Design Considerations for Speech Needs

Communication is one of the key elements for engaging in a game based interaction. Individuals typically use verbal and non verbal forms of communication while engaged in activities. This section covers considerations for providing alternative methods to communicate for individuals with speech needs to be able to enjoy the interactive components of the exhibit.

• Ensure that speech input is not required, and included only as a supplementary / alternative input method

• Allow a preference to be set for playing online multiplayer with players who will only play with / are willing to play without voice chat

• Support text chat as well as voice for online multiplayer

• Provide visual means of communicating in multiplayer

• Base speech recognition on individual words from a small vocabulary (eg. ‘yes’ ‘no’ ‘open’) instead of long phrases or multi-syllable words

• Base speech recognition on hitting a volume threshold (eg. 50%) instead of words

• Use symbol-based chat (smileys etc)

1. Language and accent & tone detection: Ability to change language of AI chat bot in relation to the speakers language. Ability to understand the accent and tone (happy, sad, angry etc…) of the user. The following example shows how an AI detects the user speaking english, and switches the language output to the next question.

2. Use of Symbolic Communication - This feature allows users with limited speech to quickly communicate with others in a game or to interact with an artificial intelligence speech bot interface. The following symbols are standard for communication during gameplay, and could be used as way for users to quickly interact with others in a social environment:

Example of products that utilizes this technique: Talk Tablet and Proloquo

1. Hand Gesture Tracking - Sign language or hand gesture detection for artificial intelligence interfacing interactions.

2. Keyboard/ controller response capability: Have the ability to interact with the speech component of the digital experience with the use of a touchscreen button, controller or keyboard. This gives the users options on how to interact with the scene. An example for an AI speech bot, is to have the buttons “Yes” or “No” available for quick response, and a keyboard to type in complex responses.