The "flagship" project in my lab is the guide robot - a robot that can guide visually impaired people. The project includes several research questions such as: how to recognize the goal of a handler walking with a robot? How to handle cases where there is a conflict between what the handler wants to do and what the robot thinks they should do? In other words, when and how should such a robot intelligently disobey a command? How to navigate in a social manner around other people? How to learn and adapt to a new handler? For more information about this idea, see the "seeing-eye robot" paper.
Each of the questions above can developed into a research project. Below are some examples for potential or ongoing research in my lab:
A goal recognition project which aims to learn what are the set of potential goals a person might have from raw images or from demonstrations, using reinforcement learning. The project involves work in simulation and later on in the real world. This project will rely on a framework presented in the paper "Goal Recognition as Reinforcement Learning".
An ongoing project where we aim to make an AI that can suggest a good curriculum for a person to learn a motor skill, like controlling the joystick when playing a specific video game. This is an interdisciplinary project with collaborations with Mechanical engineering, Neuroscience, and Computer science (which is where our contribution takes place). A summary of what we accomplished so far can be found in: "Kinematic coordinations capture learning during human-exoskeleton interaction"
In this new project, we are interested in formulating disobedience using various planning representations. We aim to explore - how would disobedience be different, if agents have partial observability of the environment? If they have asymmetric roles? Would users be willing to accept an AI that disobeys or that is less sycophant than ChatGPT? The foundations of this work have been laid down in "Artificial Intelligent Disobedience"
Current LLMs and other foundation models provide amazing breakthroughs in automation, but at the cost of hallucinations that significantly hinders the reliability of these models. A key component to improve these models is by augmenting them with epistemic reasoning, where each response is accompanied by confidence levels. Using this confidence, the model would be able to respond "I don't have high confidence in my response". This is a new project.
Current vision-language models excel at detecting visible objects, but they lack the commonsense reasoning needed to infer where everyday items are stored, especially when those items are hidden from view. This project introduces a new benchmark, the Stored Household Item Challenge, to evaluate models' ability to predict likely storage locations based on visual context and semantic priors. As a first step, we proposed NOAM, a language-driven pipeline that reframes visual inference as structured reasoning, enabling large language models to infer hidden object locations.
Animal-Human-Machine (AHM) teams are a type of hybrid intelligence system wherein interactions between a human, AI-enabled machine, and animal members can result in unique capabilities greater than the sum of their parts. This project will explore AHM teaming, investigate how to better mediate between the different teammates, and evaluate the dynamics between the team members. "Birds of a Different Feather"
Research is fluid, and it typically ends due to resource constraints (project termination, student graduation) rather than because the problem is completely solved. Interesting extensions can emerge from these past projects.
The goal of this project is to study the compliance of social robot navigation within different cultures. As this is a new project, there are no available papers, but you can look at the SCAND dataset to see the type of navigational behavior which will be explored in this project. The data collection process of SCAND will be replicated in Israel to examine the effect of cultural differences. Socially CompliAnt Navigation Dataset
An ongoing project I'm working on is collaboration with new teammates (e.g. like when you play football with players you never played with before). This topic is generally called ad-hoc teamwork (closely related to zero-shot coordination) and specifically, I'm working with a student who's looking at potential communication channels that can be used to improve such interactions. The first paper in this project is "A penny for your thoughts".
This project investigates how people perceive the behavior of a dog-like robot if it "wears" a vest like seeing-eye dogs, being held with a leash, etc. This is an ongoing project at the University of Texas at Austin, and the work will be replicated in Israel to examine the effect of cultural differences. The research methodologies used in the project will the similar to the ones in this paper "Using human-inspired signals..."
This project focuses on understanding the intentions of a person blocking a robot. This was a collaboration between academia and industry, with robotics companies that aim to integrate the algorithms developed in the lab in their robots. This work already yielded NIMBLE (Navigational Intentions Model for BLocking Estimation), a general framework for blocking detection.