Why Preference-based Planning

Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in collaborative roles. Although recent studies on embodied intelligence have advanced significantly, they typically adopt generalized approaches that overlook personal preferences in planning. We address this limitation by developing agents that not only learn preferences from few demonstrations but also learn to adapt their planning strategies based on these preferences. Our research leverages the observation that preferences, though implicitly expressed through minimal demonstrations, can generalize across diverse planning scenarios. To systematically evaluate this hypothesis, we introduce Preference-based Planning (PbP) benchmark, an embodied benchmark featuring hundreds of diverse preferences spanning from atomic actions to complex sequences. Our evaluation of SOTA methods reveals that while symbol-based approaches show promise in scalability, significant challenges remain in learning to generate and execute plans that satisfy personalized preferences. 

Preference-based Planning Benchmark

We introduce Preference-based Planning (PbP), a comprehensive embodied benchmark built upon NVIDIA Omniverse and OmniGibson. PbP provides realistic simulation and real-time rendering for thousands of daily activities across 50 scenes, featuring a parameterized vocabulary of 290 diverse preferences. Tasks in PbP mirror real-world watch-and-help scenarios, where an agent observes a few demonstrations of a user performing tasks that reveal preferences. The agent must then complete similar tasks in different setups while adhering to the demonstrated preferences. 

Preference-based planning comprises two key components: few-shot preference learning of user preferences and subsequent planning guided by these learned preferences. Since humans, even infants, can naturally detect others’ preferences from limited decisions, and collecting extensive personal demonstrations is impractical in daily life, we formulate this as few-shot learning from demonstration.

(Top left) We propose a hierarchical organization of user preferences. Our framework organizes preferences in a three-tiered structure, visualized through sunburst diagrams: (a) Action level, (b) Option level, (c) Sequence level. Each diagram’s hierarchical structure branches from general categories to specific instances, revealing detailed preference patterns upon closer inspection. 

(Top right) We also show an example of a demonstration in PbP. The robot in the demonstration is executing the task “Pick Apple from Fridge and place on Table”. 

• Top: A third-person view video provides an overhead perspective of the entire scene. 

• Middle: The bird’s-eye-view map displays the robot’s relative position within the scene. 

• Bottom: The egocentric video captures the robot’s first-person observations during task execution. 

• Text: The per-frame action annotations contain Omniverse object IDs, which ensure each object reference is unique and enable the model to identify specific objects precisely.

Experiments: Preferences as a Valuable Abstraction

We further demonstrate that incorporating learned preferences as intermediate representations in planning significantly improves the agent's ability to construct personalized plans. These findings establish preferences as a valuable abstraction layer for adaptive planning, opening new directions for research in preference-guided plan generation and execution.

We also show our analysis of test samples in direct and generalization settings. Lines represent distinct scenes, with grid colors indicating different sample statuses. There are two key findings: (i) Preference learning performance correlates with scene characteristics, with certain scenes proving consistently challenging across both conditions. (ii) While direct cases show better performance overall, failure patterns differ between conditions, particularly for vision-based models. This suggests models rely heavily on visual context consistency–including object arrangement and scene layout–for accurate predictions, indicating potential superficial learning rather than true preference understanding. Symbol-based reasoning maintains robust performance across varied scenes due to the general nature of predefined preferences, whereas vision-based models’ strong dependence on specific visual contexts limits their generalization capability.