Mid-air bare-hand gesture interaction for smart TV

Goal: To design “good” gestures for TV control

My work:

1) Literature research to understand current status and define requirements for “good” gestures 

• What functions do smart TVs have?

• How do users carry out those functions?

• Comparison to the alternative interaction methods (remote, voice)

• Prioritized functions to replace with gesture

2) Facilitation of internal discussion

3) Gesture elicitation study

Questionnaire & Interview with internal participants (N=20)

4) User acceptability verification study

• Online questionnaire with external participants (N=747  in Canada) 

• Online questionnaire in China (N=300) to check cultural effects

Outcomes:

• 129 task-gesture combinations were generated in total, inducing 72 distinct gestures.

• Proposed the most accepted gesture-task combinations based on the online survey both in Canada and China.

Jiang, H., Chignell, M., Mizobuchi, S., Farhadi Niaki, F., Liu, Z., Zhou, W., & Li, W. (2021, June). Demographic Effects on Mid-Air Gesture Preference for Control of Devices: Implications for Design. In Congress of the International Ergonomics Association (pp. 379-386). Springer, Cham. https://doi.org/10.1007/978-3-030-74614-8_47

Goal: To design GUI for gesture-based TV control

My role: Evaluate the prototype GUI and feedback design

My work:

• Conducted user testing with prototype system 

• Performance analysis on how accurately/quickly user can change volume/move video frames

• Behavior observation with gaze analysis

• Interview studies

Outcomes: 

The test result showed that few participants understood the meaning of the icon and didn’t watch it while operating the gesture input. I suggested feedback icon design and location to improve the visibility & recognizability.

Goal: Identify good CD gain type and gain

My work: 

• Development of a research roadmap, experimental plan, supervision of collaborative researchers, supporting the data analysis, co-authoring of reports and papers.

• The preparation of experimental equipment and programs, as well as the execution of the experiment, were carried out by intern students. 

Outcome:

• Recommended parameter setting to the development team, Patents

Zhang, F., Mizobuchi, S., Zhou, W., Khan, T. A., Li, W., & Lank, E. (2021). Leveraging CD Gain for Precise Barehand Video Timeline Browsing on Smart Displays. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part IV 18 (pp. 72-91). Springer International Publishing. https://doi.org/10.1007/978-3-030-85610-6_5

Jay Henderson, Ali Neshati, Sachi Mizobuchi, Wei Zhou, Daniel Vogel, and Edward Lank. 2023. Interaction Region Characteristics for Midair Barehand Targeting on a Television. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 180, 1–7. https://doi.org/10.1145/3544549.3585877