Edward S. Hu*¹, Jie Wang*¹, Xingfang Yuan*¹, Fiona Luo1, Muyao(Lilian) Li¹, Gaspard Lambrechts², Oleh Rybkin³, Dinesh Jayaraman¹
¹ GRASP Lab, University of Pennsylvania ²University of Liège ³BAIR Lab, University of California, Berkeley
*Equal contribution.
A robot's instantaneous sensory observations do not always reveal task-relevant state information. Under such partial observability, optimal behavior typically involves explicitly acting to gain the missing information. Today's standard robot learning techniques struggle to produce such active perception behaviors. We propose a simple real-world robot learning recipe to efficiently train active perception policies. Our approach, asymmetric advantage weighted regression (AAWR), exploits access to ``privileged'' extra sensors at training time. The privileged sensors enable training high-quality privileged value functions that aid in estimating the advantage of the target policy. Bootstrapping from a small number of potentially suboptimal demonstrations and an easy-to-obtain coarse policy initialization, AAWR quickly acquires active perception behaviors and boosts task performance. In evaluations on 8 manipulation tasks on 3 robots spanning varying degrees of partial observability, AAWR synthesizes reliable active perception behaviors that outperform all prior approaches. When initialized with a ``generalist'' robot policy that struggles with active perception tasks, AAWR efficiently generates information-gathering behaviors that allow it to operate under severe partial observability for manipulation tasks. Website: https://sites.google.com/view/rwrl-ap/home
We evaluate AAWR in 8 different manipulation tasks in sim and real.
We also ablated dataset quality in our hardest task, Complex. We found that a clean demonstration dataset greatly boosts all approaches. Our method still does the best. See Appendix F.8 for details.
Below, we showcase videos of the policies in the 4 real world active perception experiments.
vert_pi0_f.mp4First, we show representative Pi0 behavior in such searching tasks. It does not efficiently search, and in many cases will never fully explore the scene. This results in very low success rates and very long search times.
AAWR
hard_aawr_s1.mp4AWR
hard_awr_s1_rect.mp4BC
hard_bc_s1_rect.mp4hard_aawr_s2.mp4hard_awr_f1_rect.mp4hard_bc_f1_rect.mp4hard_aawr_f1.mp4hard_awr_f2_rect.mp4hard_bc_f2_rect.mp4hard_vlm_f.mp4hard_oplp_f.mp4We can see that although exploring the cabinet on the left in general is a difficult task, only AAWR showed consistent ability to explore the cabinet and success.
AAWR
dual_aawr_s1.mp4AWR
dual_awr_s1.mp4BC
dual_bc_s1.mp4dual_aawr_s2.mp4dual_awr_f2.mp4dual_bc_f1.mp4dual_aawr_f1.mp4dual_awr_f3.mp4dual_bc_f2.mp4dual_vlm_f.mp4dual_oplp_s.mp4AAWR
vert_aawr_s1_rect.mp4AWR
vert_awr_s1_rect.mp4BC
vert_bc_s1_rect.mp4vert_aawr_s2_rect.mp4vert_awr_f1_rect.mp4vert_bc_f1_rect.mp4vert_aawr_f1_rect.mp4vert_awr_f2_rect.mp4vert_bc_f2_rect.mp4vert_vlm_f.mp4vert_oplp_s.mp4We can see that both AWR and BC tend to move away even when the pineapple is quite visible in the wrist camera, while AAWR learns to "lock on" to the target object. AAWR also searches the top and the right side (far way from initial pose) of the bookshelf more closely.
AAWR
duck_aawr_s1.mp4AWR
duck_awr_s1.mp4BC
duck_bc_s1.mp4duck_aawr_s2.mp4duck_awr_f1.mp4duck_bc_f1.mp4duck_aawr_f1.mp4duck_awr_f2.mp4duck_bc_f2.mp4duck_vlm_f.mp4duck_oplp_f.mp4AAWR (offline)
AWR (offline)
20250413_113550.mp4poking.mp4AAWR (online)
AWR (online)
BC
succ.mp4succ.mp4noisy_succ.mp4retry_2.mp4holding.mp4noisy_miss.mp4We can see that BC is very noisy and has high variance, leading to low pick and pick rate. Offline AWR is more stable, but has lower accuracy and sometimes can't maintain a firm grasp. This is mitigated but not solved by the online training. Offline AAWR is stable and accurate, but also sometimes can't maintain its grasp. However, after the online training, this problem was gone, and additionally, the policy shows a reliable retrying behavior.
aawr_succ_camopick.mp4awr_fail_camopick.mp4bc_fail_camopick.mp4In this visually challenging task, the robot must pick up a tiny, hard to see marble from only an 84x84 RGB image.
aawr_succ_fopick.mp4awr_fail_fopick.mp4bc_fail_fopick.mp4Even in the fully observable case where privileged information may be redundant, AAWR outperforms non-privileged baselines like AWR and BC. We can see that only AAWR learns to precisely pick up the block, whereas AWR and BC struggle to place the gripper directly on top of the block.
koch_wrist_awr_converged.mp4koch_distillation_eval.mp4koch_estimator_failure.mp4Only AAWR is able to learn to scan the workspace, and is able to pick up blocks with near 100% success rate. Distillation and VIB learn suboptimal strategies of directly approaching the center of the workspace, which works in many cases but not when the block is initialized away from the center.