The Quality of HCI in Connected Environments (KETI, DRAPER)

This project aims to understand users-on-the-go in connected environments and to improve the quality of their ubiquitous HCI experience by enhancing machine intelligence to be more human-centered. Much of the preliminary work for this project has been completed, and the ongoing work for this project will develop in two phases:

Figure 11. Preliminary work for Phase 1 and Phase 2 to explore the proposed scope of this project.

In Phase I, I study the Internet of Things, which is combined with wearable technologies. The goal of this phase is to understand user interaction with connected devices, which are either wearable for users (e.g., smartwatch) or embeddable for systems (e.g., tiny versatile sensors). Based on this understanding, I explore user interface schemes that can enhance the quality of interaction. The specific tasks for this area include the development and fusion of contactless and contact user interfaces (especially input devices), the creation of a conceptual framework, the design of multisensory interaction schemes, the construction of a user interface framework, and the development of test applications.

As preliminary research for this first phase, I currently serve as the principal investigator on an international project about wearable technologies (“Development of UI/UX Technology to Overcome the Limitations of Wearable Device UIs”, Project period: 2014 Nov – 2017 Oct). In this project, we test and evaluate the usability of a series of advanced-concept user-interface prototypes for wearable devices (e.g., smartwatches), improve their usability through iterative design, and identify the strengths and weaknesses of each prototype in order to suggest design implications for a more usable user interface and user-interaction method (Figure 11a).

In Phase II, I study Human-in-the-Loop Cyber Physical Systems, which are combined with context-aware and user-aware machine intelligence. The goal of this phase is to present machine intelligence that can understand how the states of various end-users change in real time and that is robust to various situational events, thereby helping people interact seamlessly with connected systems in connected infrastructure.

As preliminary research for this second phase, I participated in a research and development project about immersive situation awareness (“User-Centric Study of Human Interaction with Visual, Audio, and Haptic Presentation Systems”, 2015 Jul – 2016 Jun). In this project, we proposed a generic contextual framework that can support immersive situation awareness for 26 different scenarios (Figure 11b). The framework consists of modules related to sensing; a context identifier; an information provider; presentation mapping; processors for value estimation, cost estimation, prioritization, intervention determination, and filtration (candidate info combo); and an actuator controller. We have verified and refined the operation of the framework through thorough literature reviews and a set of human subject experiments.