AI has become increasingly capable of understanding and reasoning about text-based tasks, and even performing agentic tasks on behalf of humans. However, visual understanding, spatial reasoning, and physical interaction remain highly complex and expensive for current AI systems, particularly when operating in real-time. Thus, developing AI that can contribute meaningfully to physical interaction presents a significant challenge. Current physical AI systems (AI models trained on 3D capture of humans and objects interacting in the world) operate slowly, handle only simple tasks, and remain far from being practical or safe for real-world deployment in many cases. To address this, augmenting physical AI with efficient human-AI communication leverages the strengths of both parties, given the current maturity of AI technology.

In this summer scholarship project, we aim to empower AI-driven agents (robots) by integrating human operators and decision-makers, immersed in a virtual environment, directly into the control loop when needed. The goal is to enable operators to make safe, real-time decisions while delegating cognitively simpler tasks to the AI, thus limiting human operators' intervention on every step. Specifically, we will investigate how decision-making problems, derived from both the physical environment and the AI agent, can be represented effectively to human operators. We will also explore which interactive techniques best support collaboration and assess the overall effectiveness of the resulting system.

We believe that human-physical AI collaboration will become a central focus of HCI research in the coming years. Through this work, we will identify generalisable principles across different applications, uncover key challenges, and generate insights and practical experience for future research. 

Required Skills:

• General software engineering and programming knowledge

• Unity or Unreal

• Robotics Principles

Other requirements: 

• Please note that, due to the nature of this project, the successful candidate will be required to work on-site at Monash University's Clayton campus.

Please ensure you possess the required skills, meet the summer scholarship 2026 eligibility criteria, and submit your official application through Summer Scholarships application.