MO-DDN: 

A Coarse-to-Fine Attribute-based Exploration Agent for Multi-object Demand-driven Navigation

Hongcheng Wang, Peiqi Liu, Wenzhe Cai, Mingdong Wu, Zhengyu Qian, Hao Dong

Bilibili      YouTube     Paper      Code 

Abstract

The process of satisfying daily demands is a fundamental aspect of humans' daily lives. With the advancement of embodied AI, robots are increasingly capable of satisfying human demands. Demand-driven navigation (DDN) is a task in which an agent must locate an object to satisfy a specified demand instruction, such as ``I am thirsty.'' The previous study typically assumes that each demand instruction requires only one object to be fulfilled and does not consider individual preferences. However, the realistic human demand may involve multiple objects. In this paper, we introduce the Multi-object Demand-driven Navigation (MO-DDN) benchmark, which addresses these nuanced aspects, including multi-object search and personal preferences, thus making the MO-DDN task more reflective of real-life scenarios compared to DDN. Building upon previous work, we employ the concept of ``attribute'' to tackle this new task. However, instead of solely relying on attribute features in an end-to-end manner like DDN, we propose a modular method that involves constructing a coarse-to-fine attribute-based exploration agent (C2FAgent). Our experimental results illustrate that this coarse-to-fine exploration strategy capitalizes on the advantages of attributes at various decision-making levels, resulting in superior performance compared to baseline methods.

Video

MoDDN - 05.mp4

Overview

Main Results


 

Ablation Study


Q1: Is selecting waypoints by attribute feature similarity scores better than FBE, LLM and CLIP features' similarity scores? 

Q2: Do attribute features also work in the end-to-end fine exploration modules? How about replacing the fine exploration module with VTN and ZSON?

Q3: Do VQ-VAE losses and codebook initialization contribute to experimental results?

Q4: Can adjusting the weights of basic and preferred scores affect agent behavior?

Block Score Visualizations

 

Contact Us

Citation

@inproceedings{wang2024mo,

  title={MO-DDN: A Coarse-to-Fine Attribute-based Exploration Agent for Multi-object Demand-driven Navigation},

  author={Wang, Hongcheng and Liu, Peiqi and Cai, Wenzhe and Wu, Mingdong and Qian, Zhengyu and Dong, Hao},

  booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},

  year={2024}

}