POMDP-based object search with growing state space and hybrid action domain

Code coming soon.

Object Search Task

In this paper, we assume that the robot is operating in a home environment that consists of multiple rooms with multiple given workspaces (furniture surfaces) and is provided with 3D point cloud and 2D occupancy grid maps of the environment and a set of the target object’s photo. Robot is pointed to search a snack box (pink). It adjusts its base, lift, and head (red) to change its FOV, with no prior knowledge (like size and number) of non-target objects.

Methodology

We address this by framing the object-search task as a high-dimensional Partially Observable Markov Decision Process (POMDP) with a growing state space and hybrid (continuous and discrete) action spaces in 3D environments. Based on a meticulously designed perception module, a novel online POMDP solver named the growing neural process filtered $k$-center clustering tree (GNPF-$k$CT) is proposed to tackle this problem. Optimal actions are selected using Monte Carlo Tree Search (MCTS) with belief tree reuse for growing state space, a neural process network to filter useless primitive actions, and $k$-center clustering hypersphere discretization for efficient refinement of high-dimensional action spaces. A modified upper-confidence bound (UCB), informed by belief differences and action value functions within cells of estimated diameters, guides MCTS expansion.

Results