The study of behavior is important to a variety of fields, from quantifying animal behavior in neuroscience, to vehicle and pedestrian forecasting for autonomous vehicles, to realistic generation for animation and video games. Across these fields, computational models have enabled us to analyze and study behavior scalably. However, common open questions remain across fields and we organize workshops, challenges, and datasets in order to facilitate discussions for researchers across fields and to advance computational behavior methods.
What do people want to compute from observed behavior?
Description: Provide a summary of what happened, while ignoring useless details.
What are factors that could affect the description?
What does an ‘unbiased’ description look like?
How do we describe agent behaviors across different timescales and spatial scales?
Prediction: Provide an expected trajectory of what an agent will do next.
what will happen in the next milliseconds, seconds, minutes?
What happened before we started observation?
What is the probability that agent A will perform action X?
Causality: Identify the environmental variables that shape agents' behavior.
How do we identify important cues in an agent's environment?
How do we distinguish between correlation and causation on variables that affect agent behavior?
Which brain mechanism is controlling the behavior?
Motivational State: Infer unobserved aspects of an agent's state (stress, hunger, alertness) that influence the agent's actions.
What objective is an agent trying to achieve?
How does an agent's goal shape its actions at different timescales?
What cues in the environment is the agent paying attention to?
Which emotions or internal states are influencing the behavior of an agent?
Generation: Provide a description of how an agent would perform a behavior in a given environment.
What algorithms are consistent with an agent's observed behavior?
How might an agent modify their actions to improve their performance towards a goal?
How does an agent measure its performance?
Interaction: Behavior of an agent that is oriented towards another agent or object in the environment.
What stimulus caused the agent to produce the observed behavior?
How do agents interact with the environment and other agents to produce behavior?
What role is an agent playing in a given multi-agent scenario? (Cooperative, competitive, etc)
Computational challenges:
Generalization
Are there fundamental building blocks / tools to studying behavior across domains?
To what extent can behavioral models generalize across domains/organisms/settings?
Accuracy
What description or representation of behavior captures important information for downstream analysis?
Does the model produce consistent behavioral descriptions?
Interpretability
How does the model help researchers answer questions on behavior?
What behavior descriptions and representations are most semantically meaningful for domain experts?
Encoding Knowledge
What inputs from domain experts are needed to connect the model to questions of interest?
What is the most effective way to encode knowledge beyond increasing the amount of data annotations?