Accessibility testing in VR - part two


Introduction

In this post I’m sharing my approach to accessibility testing in Virtual Reality (VR). This is part two of my VR accessibility posts, last time I wrote about why it’s important.

Think of it more as an exploratory checklist rather than a rigid test script.

First pass

I like to do a first pass of a new experience with some of the functionality disabled, this helps me to identify where there maybe problems for people with certain accessibility needs.

For the first pass, my setup is as follows:
      Set application to greyscale[1] where possible
      Turn sounds off or volume down
      Sit on a non-swivel chair (or sofa or bean bag)
      Avoid reading any text or set text to a language that you don’t understand[2]
      Use a single controller (if possible)
      Disable haptics where possible[3]
      Set scale to “standing only” [Vive or Rift or other 6 DoF headsets?]

Attempt to use the application as “normal”, but pay attention to ease of understanding, navigation and interaction:
      Is it hard to distinguish the grey content from the slightly darker grey content - this might highlight where someone with poor vision or colour blindness may struggle
      Is there sound or haptic feedback obviously missing - note this for comparison with the full experience
      Where interactions are difficult or uncomfortable: can all elements be interacted with, can you reach them without standing or bending over? How easy / necessary is turning or looking behind you?
      Can the experience be completed without the need to understand the text? What about set up options?
      Is the experience easily navigable and operable with a single controller?
      For a 6DoF experience, is the standing only setup missing anything?

If your VR experience doesn’t work on a sofa then it may not be accessible for all users

Comparison

Next run through the same experience with the features enabled:
      Return to direct mode (disabling greyscale)
      Re-enable sound and haptics
      Set to default language
      Stand
      Set to room-scale (for 6DoF experiences)

Compare this experience with the previous one.
      Is there important information presented this time that was not presented in the first run through?
      Visual / colour
      Auditory
      Haptic
      Text based
      Are there options or interactions available that were not obvious the first time?
      Does the room based experience offer more interaction possibilities that the standing-only scale did not?

This comparison will highlight where the experience could be lacking for people with specific accessibility needs.

Focus on specifics

Following on from the first pass and comparison, a more thorough analysis may reveal further potential issues or opportunities for improvement. The experience of accessibility testing for web sites has taught us that improvements made for people with disabilities are generally more usable to everyone.

What follows is a list of points to consider, grouped into themes. Many of these have been taken from the web content accessibility standard where appropriate, others are particular to VR. Whilst the target here is inclusion, many of the recommendations are likely to improve usability for all.
VR experience in greyscale
If you needed to push the red button it is more problematic if you can’t distinguish between the colours
[Image source: Job Simulator, converted to greyscale]


Visual

Test
Rationale
Recommendations
Flashing
Watch for any areas in the experience where there is flashing or strobing.
Flashing and strobing can cause seizures. Whilst most users will be aware of their own vulnerability, images in VR can potentially occupy a much higher percentage of the visual field and can therefore cause problems for those with a very high threshold that may not be aware of any issues previously.

[WCAG 2.1 has recommended limits to flashing but these are for 2D web based interactions and may not be translatable to 3D environments - the safest option is to avoid flashing.]
There is no flashing / strobing within the experience.

Or warn the user and give them the option to remove flashing.

If the flashing effect is optional, its removal should not negatively impact the experience for these users.


Use of colour


The easiest way to check for this is to set the system to greyscale.

To change to greyscale in Vive:
      Disable direct mode (this will treat the HMD as another monitor)
      Adjust properties from the graphics card settings (reduce saturation)
      Turn Steam VR off and on again
      Should now be greyscale
Note: this setup can cause performance issues such as dropped frames, so it’s important to be aware of potential cybersickness implications for your tester
If interaction is difficult or poorer when in greyscale, this is an indication that it may cause problems for people with colour-blindness.
Where colour is used to differentiate an element, consider including an alternative method of identification or providing options for colour-blind users.
Contrast
Check that text, interactable objects and anything vital for navigation has sufficient contrast.

This can be measured by taking screenshots within the experience using the device’s screen capture then measuring the contrast. My preferred method is to:
      open the screenshot in a browser
      Use Colorzilla to capture the foreground and background colours
      Use Colour Contrast Checker to measure the contrast
Users with low vision may struggle to discern content with poor contrast.

Improving contrast is likely to improve the usability for all sighted users.
Sufficient contrast would achieve a contrast ratio equivalent to WCAG 2 AA compliance.

Monocular
Navigate the experience with one eye closed.

Switch eyes occasionally. The experience should be playable in the same way as when both eyes are used.


Although, some experiences display the same image in both screens on the headset others allow for binocular vision.

Users with monocular vision (either temporary or permanent) will be disadvantaged if binocular vision is a requirement.

This may also affect individuals with one eye stronger than the other.
All functionality should be available for a user with one functioning eye.

If the experience is designed to only be used by someone with binocular vision, this should be made explicit before the experience in purchased and before the experience starts.
Visual reinforcement
Observe where sound or haptic feedback is used to present meaning to the user.

This content should also be provided by some visual mechanism.
This will enable users with hearing or tactile impairment or those that do not wish to use audio or haptic feedback to have the same cues.

Visual cues can also enhance the experience for all users.
Where visual signifiers do not fit with the desired aesthetic of the experience these cues could be optional (e.g. closed captions)

Captioning or speech bubbles could feature real-time translation as well as assisting those with hearing difficulties
[Image source: Windows Mixed Reality]


Auditory

Test
Rationale
Recommendations

360 spatial sound
Using hearing alone, the user is able to determine the approximate direction and distance of a sound’s origin.
This provides a more realistic experience for all adding to immersion.

It may also enable the experience to be used by those with limited or no vision.
Utilise full 360 spatial sound enabling the user to determine distance and direction from sound alone.
Audio-visual synchrony
Examine the experience and determine if visual elements and interactions should have a corresponding sound.
This will provide a better user experience for all and add to immersion for most users


Visual (and haptic) elements have appropriate audio components where applicable.

The audio should be in sync with the visuals
No sound
In contrast to the above. Can the experience be operated without sound?
This will enable those with limited or no hearing to use the experience. It will also allow any user to operate in an environment with noise pollution or with the sound turned down low.
Consider other factors to draw attention, maybe using lighting or visual cues.
Closed captioning
Observe where any non-visual content provides meaning or atmosphere to the user.

Where this is not provided using visual cues, closed captioning should be made available.
This provides a better experience for those with auditory impairment.

It may also improve the experience for those with low vision and certain cognitive impairments.

Closed captioning option is available for:
- visual environment
- interactive elements
- navigational and informational elements
- ambient sound effects
- essential dialogue
- non-essential "mood" dialogue
- haptic feedback
- in scene changes.
Balance
Listen to the audio with stereo headphones paying particular attention to balance and orientation.
360 spatial sound can provide a lot of benefits to how the experience is perceived, but should not be used in a way to intentionally cause disorientation or loss of balance as a game mechanism.
Audio is not used in a way to cause disorientation or loss of balance.




Haptic


Test
Rationale
Recommendations
Haptic reinforcement

Observe where there may be opportunities to enhance the experience with the use of haptic feedback
This provides a more realistic experience for all.
Audio and visual feedback is reinforced with haptic feedback where appropriate















[1] Using a phone’s accessibility features for mobile VR, It’s sometimes possible in Vive but slightly more complicated and involves disabling “direct mode”

[2] Make sure you record how to change the language back ;)
[3] Using a phone’s accessibility features. May not yet be possible in Vive without dismantling the controllers

Comments