The research and prototyping process revealed the following themes, closely tied to the pain-points presented below.
Non-sequential spatial access
Management of cognitive and sensory load
Perceptually-distinguishable information (and control)
Cross-modal Redundancy
Relevant Content Curation
Orientation and feedback: Without continuous audio feedback, or feedback available upon request, Blind or Low-vision (BLV) users of Virtual Reality interfaces quickly lose track of where they are, and what is around them.
Inability to “rehearse”: Lack of perceivable spatial relations and high-level summaries of content harms the ability of BLV users to “rehearse” interactions for real spaces remotely or plan in advance.
Lack of awareness of system status: Related to the lack of perceptual feedback indicated above, BLV users often have little to no information to foster awareness of a system’s status and possible actions that may be taken when using ICTs.
High cognitive load from unnecessary repetition: Information foraging (Pirolli and Card, 1999) efforts are frequently rendered inefficient and mentally taxing due to repetitions of the same information in the same sensory modality (e.g., navigating menus with screen readers that require going through the same sequences every time to reach specific items, repeated labels of features that the user is already aware of from context, etc.)
Need for navigable, searchable lists of all content: In order to forage directly, rather than through exploration of unknown structures or spaces, users need a list of everything that they could find that can be quickly scanned for relevant items.
Information overload: Sheer quantity of information (even if not repetitive) is often encountered when using ICTs, causing overload if there is no additional organizational structure or limitation of the quantity being pushed on the user at any given moment.
Clashing Auditory Input: One significant pain point that virtual meetings address is the overwhelming auditory input experienced by screen-reader users. They often contend with clashing voices from video participants and simultaneous announcements from the chat. This reportedly led to sensory overload and difficulty in focusing on the meeting's content.
Interruptive Chat: The chat feature, intended for real-time communication and collaboration, was often disruptive for screen-reader users. Many participants chose to disable the chat entirely to avoid interruptions and maintain focus on the spoken discussion. However, this decision often results in missed messages and a sense of disconnection from the conversation.
Temporal Synchrony: Screen-reader users frequently struggle with temporal synchrony when reviewing messages in the chat. Going back to catch up on previous messages while simultaneously following the ongoing conversation poses a challenge. This lack of synchronisation can lead to mis-timed responses, and disconnection from the meeting flow.
Message Notification Control: Another pain point is the lack of agency over message notifications. Screen-reader users often found they had little control over how and when they received notifications, leading to interruptions and distractions during the meeting.
Adapting to Hotkeys: Screen-reader users rely heavily on keyboard shortcuts and hotkeys for frequently used commands. Memorising these shortcuts requires cognitive effort and practice, made worse by differing hotkeys for the same function, across platforms.
Getting to interactive content: due to a lack of spatial topological context and needing to press through negative space or swipe/tab through non-useful information.
Tactile Sheet Overlay on an iPad for Zoom
The recommendations were developed through prototypes that defined the following paradigms. These paradigms can be thought of as alternatives to the current ubiquitous paradigm of the Graphical User Interface (GUI). You can explore these recommended paradigms in detail here.
Recommendation I: 2D Navigation and Detection of Information From Range
1. Spreadsheet Navigation Format
2. Speech Labels
3. Binaural Audio Features
4. “Tracing”
5. “Teleportation”
Recommendation II: Range of Information Granularity
1. Hierarchical linked layers
2. Contextual Non-Spatial “Index”
Recommendation III: Foster Mental Model Construction and Recollection
1. “Guided Tours”
2. “Landmarks”
3. Non-Linguistic Audio Cues
Recommendation: Provide Methods to Convey Spatial Relations for Interfaces
1. Provide Options for Interface Overviews
2. Provide diverse formats and representations
3. Make content distinguishable
Recommendation: Minimise User-strain for Repetitive Functions and Notifications
1. Provide options for haptic feedback (vibration) indicators
2. Prioritise Consistent and Recognizable Patterns for User Interactions
3. Allow for spatially-independent or unique buttons
Recommendation I: Dialogical Interaction
1. Curation of content in feedback loops for relevance
2. Provide feedback on system status
Recommendation II: Customizable Output
1. Creation of relevant output for diverse audience needs via text and speech
Recommendation III: Effective translation of visual information to text and speech
Recommendation IV: Voice Interaction Interfaces
Curious about implementing these recommendations into your workflow or products?
Explore how these recommendations come together in practice, to create a cross-sensory design tool