Overview : 2019-2020 at Huawei. With the booming Internet of Things (IoT), an increasing number of appliances are becoming controllable. This raises the question of how we should interact with these devices, moving beyond the traditional remote controller.
The team comprises a Project Manager (PM) and a varying number of software engineers, typically including 3 to 8 interns. Additionally, there are 3 to 6 Human Factors (HF) and User Experience (UX) researchers, as well as a UI design team based in the headquarters.
Project goal: Provide bare-hand mid-air gesture control for new smart TV product
My role: Support product development from a human factors perspective leading HF/UX team
First, we defined what constitutes a 'good gesture ' --- it should be easy to guess, easy to learn, and easy to perform. To design such gestures, we conducted a gesture elicitation study, where participants were asked to freely come up with gestures for executing commands on the TV.
Goal: To design “good” gestures for TV control
My work:
1) Literature research to understand current status and define requirements for “good” gestures
What functions do smart TVs have?
How do users carry out those functions?
Comparison to the alternative interaction methods (remote, voice)
Prioritized functions to replace with gesture
2) Facilitation of internal discussion
3) Gesture elicitation study
Questionnaire & Interview with internal participants (N=20)
4) User acceptability verification study
Online questionnaire with external participants (N=747 in Canada)
Online questionnaire in China (N=300) to check cultural effects
Outcomes:
129 task-gesture combinations were generated in total, inducing 72 distinct gestures.
Proposed the most accepted gesture-task combinations based on the online survey both in Canada and China.
Jiang, H., Chignell, M., Mizobuchi, S., Farhadi Niaki, F., Liu, Z., Zhou, W., & Li, W. (2021, June). Demographic Effects on Mid-Air Gesture Preference for Control of Devices: Implications for Design. In Congress of the International Ergonomics Association (pp. 379-386). Springer, Cham. https://doi.org/10.1007/978-3-030-74614-8_47
Once the prototype was created, I conducted usability tests to provide feedback on issues and improvement suggestions to the development team. Since development was carried out in an agile environment with a need for quick feedback, we repeatedly conducted small-scale tests involving 6-10 participants.
Goal: To design GUI for gesture-based TV control
My role: Evaluate the prototype GUI and feedback design
My work:
Conducted user testing with prototype system
Performance analysis on how accurately/quickly user can change volume/move video frames
Behavior observation with gaze analysis
Interview studies
Outcomes:
The test result showed that few participants understood the meaning of the icon and didn’t watch it while operating the gesture input. I suggested feedback icon design and location to improve the visibility & recognizability.
This was a collaborative research project with University of Waterloo. To enhance the controllability of a gesture-based slider UI (intended for manipulating the timeline of video content), we conducted a series of experiment to investigate the performance of slider controls at various CD (Control-Display) gains, the performance of target selection with different algorithms for selection confirmation, and the comfortable regions for users to perform gestures.
Goal: Identify good CD gain type and gain
My work:
Development of a research roadmap, experimental plan, supervision of collaborative researchers, supporting the data analysis, co-authoring of reports and papers.
The preparation of experimental equipment and programs, as well as the execution of the experiment, were carried out by intern students.
Outcome:
Recommended parameter setting to the development team, Patents
Zhang, F., Mizobuchi, S., Zhou, W., Khan, T. A., Li, W., & Lank, E. (2021). Leveraging CD Gain for Precise Barehand Video Timeline Browsing on Smart Displays. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part IV 18 (pp. 72-91). Springer International Publishing. https://doi.org/10.1007/978-3-030-85610-6_5
Jay Henderson, Ali Neshati, Sachi Mizobuchi, Wei Zhou, Daniel Vogel, and Edward Lank. 2023. Interaction Region Characteristics for Midair Barehand Targeting on a Television. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 180, 1–7. https://doi.org/10.1145/3544549.3585877
Outcome of the project
The study results were reported to the product team.
The gesture-enabled smart TV (Huawei X65) released in 2020
The product acquired Good Design Award (2020) from Japan Institute of Design Promotion
Filed patents:
1.Zhou, W., Mizobuchi, S., Singh, G., Saniee-Monfared, G., Wang, S., Zhang, F., Li, W. (2022) Methods and systems for providing feedback for multi-precision mid-air gestures on gesture-controlled device, WO-2022242430-A1
2.Li, W., Zhou, W., Mizobuchi, S., Saniee-Monfared, G., Liu, J., Arefin Khan, T., Veras-Guimaraes, R. (2022) Method and device for adjusting the control-display gain of a gesture controlled electronic device, US-11474614-B2
3.Zhou, W., Mizobuchi, S., Veras-Guimaraes, R., Saniee-Monfared, G., Li, W. (2022) Methods and systems for multi-precision discrete control of a user interface control element of a gesture-controlled device, US-2022197392-A1