Introduction

Multi-robot interactions will be a complex part of any fully autonomous team deployed in the field in the future. Additionally, tasks which are in remote locations (search and rescue, extraterrestrial exploration) or have a large number of agents (autonomous traffic routing, air traffic control) will not be able to explicitly communicate between all teammates. With constraints like this, it is imperative that agents interacting as a team must be able to account for the actions of a teammate with little to no explicit communication.

There are two subtasks for an independent robot inside a team to accomplish before this can happen. First, the robot must determine what the intent is of the teammates. This could be through direct observation, such as by seeing how a teammate moves across a forest, or it could be through sporadic communications of an teammate providing brief status updates. After the intent of the other agent is known, the robot must now incorporate this knowledge into its decision-making process.

In a competitive game such as catch, a robot which can measure the intent of the opponent can proactively account for the opponent's actions. This provides a competitive edge in high speed competitive domains, where planning around the opponents actions allows you to compensate for their high speed actions inherent in games like catch.

The goal of this research is to extend upon dynamic, adaptive opponent modeling so that opponent models can be used for both human and robotic opponents. A major difficulty for this lies in the inexpressiveness of a robotic opponent. Using purely observational techniques to attempt to predict the immediate intent of the opponent will not work on inexpressive robots. Thus, I propose directly modeling the opponent and creating a general opponent model which can be fine-tuned mid-match to capture the opponent intent and predict actions.

Background

Adaptive decision-making is a critical task for intelligent agents in teaming or competitive tasks. From competitive games like poker [4] and soccer [5] or interactive team tasks like human robot interaction [3] and multiagent negotiations [1], agents interacting with external intelligent actors can benefit by modeling and predicting the external actors' behaviors.

Explicitly modeling external agents has been done in these works through Markov decision processes (MDP) [3], reinforcement learning agents [2,4], or statistical methods like Gaussian processes [1] and explicit, domain specific models [5]. These explicit modeling methods have various benefits and drawbacks. In general, they focus on either rapid adaption to the changing environment [4,5] or on operating with unstructured interaction protocols [1,3].

The difficulty of robotic catch is that both operating regimes are required in order to effective play and win against an opponent. There is no set way in which an opponent will throw a ball, unlike playing cards in poker, and the behaviors may change wildly during the game, unlike much teaming exercises.

Proposed Methods

Evaluation Methods

Having refined the goal to only predict when the throw will occur, the evaluation metrics will look at the prediction error. Predicting a throw later than it occurs will be penalized more heavily, as this means the robot is predicting a throw after it has occurred. In this game of catch, it is better for the robot to react early than to react late.

I will use a prediction error which squares overshooting predictions and linearly considers undershooting predictions.

With Delta t = t_{actual} - t_{predicted}, the error for throw i is:

Research Question

This research will look at how precise timing predictions for human actions, in this case throwing the ball, can be performed in timing-critical domains.

Preliminary Data

Experiments

To test the time-guessing state, I first created an estimate of the ending arm position of a right-handed overhand throw. 30 ending positions were averaged into a single position, which was used as the end-arm reference hand-elbow transformation.

A single set of throws was used to create time estimates till the throw for each state. Then 3 additional sets of throws was calculated with this model, to test the generalizable of the state timings. The error function presented in Section~\ref{sec:eval} is used as the error metric in the results.

Discussion & Future Work

With the errors detected in Trials 2 and 3, the state prediction times used is only accurate at the initial guess. This is most likely due to the higher tolerances in the times as the throw progresses, due to the small magnitude of the time remaining in the Last Extension and Throwing states. Interestingly, the arm position calculations did not catch the Throwing state in the last two trials. However, this state may be unnecessary as the errors in the Wind Up time prediction are relatively low, and show premise that the final throw may be predicted through the initial windup.

This somewhat satisfies the initial research hypothesis: state-transition based methods show premise in predicting the time of a human action. However, identifying an adequate timeout still needs some work, though I observed the state identification in the initial windup was fairly accurate.

In the future, this work can be improved by incorporating velocity of the human arm into the time prediction. This would enable a wider range of throws to be classified using state-transition methods. Additionally, new throws or human actions could be mapped into state-transition graphs which would broaden this application to many complex, high-speed human action. Again, the key takeaway is that identifying the start of the action results in the most accurate time predictions

References