About

In recent years, heads-up computing has emerged as a promising paradigm, shifting the focus from traditional device-centric interactions to more fluid, context-aware systems that align with natural human behaviors. The core aim of heads-up computing is to maintain users' situational awareness, allowing them to interact with digital systems without losing focus on their physical environment. This workshop focuses on designing interactive multimodal information retrieval and access in such environments, leveraging the unique advantages of wearable and immersive technologies.

1. Multimodal Interaction in Information Retrieval and Access:

The evolution of information retrieval and access in the heads-up computing paradigm hinges on the integration of multiple modalities. Unlike traditional methods where users must engage through a single interface (e.g., typing on a screen), multimodal systems combine voice commands, gestures, and gaze tracking to allow users to query and access information. This enables a more natural and intuitive interaction style, essential for heads-up devices, where users may be engaged in other physical tasks (e.g., walking, driving, etc.).

2. Designing for Interactive Information Retrieval and Access:

The design of interactive retrieval and access systems must focus on human-centric, immediacy and relevance. Wearable systems, such as augmented reality (AR) glasses and virtual reality (VR) headsets must provide real-time results based on multimodal queries without overwhelming the user. The design should emphasize the following:

• Context-awareness: The systems should recognize and adapt to user environments and activities when forming the users’ information needs, prioritizing information based on relevance and user preferences.

• Feedback loops: Users should be able to refine and filter queries and results interactively using natural input modalities, such as hand gestures, gaze, voice, etc.

• Seamless interaction: The retrieval and access process must minimize disruptions, allowing users to remain immersed in their tasks without breaking focus to interact with the device.

3. Challenges and Opportunities:

Designing such information retrieval and access systems for heads-up computing presents several challenges:

• Privacy and security: Given the always-on nature of multimodal systems, maintaining user privacy, and securing sensitive data is paramount.

• Cognitive load: The system must balance delivering timely information without overwhelming users, particularly when multiple input modalities are simultaneously engaged.

• Adaptability: The system should be able to adjust to varying levels of user engagement and environmental complexity, ensuring robust performance in different contexts (e.g., low-light conditions, noisy environments).

However, the opportunities are vast. A successful multimodal information retrieval and access system can:

• Enhance productivity by allowing users to multitask efficiently in workspaces and public environments.

• Improve accessibility for users with impairments, enabling them to retrieve and access information in a hands-free, seamless manner.

• Advance user experience in sectors such as healthcare, manufacturing, and education by providing instant, context-relevant information.