Rebuttal Qualitative
PORT under different camera conditions. From a single input image, we visualize target views generated by four PORT variants: azimuth only, azimuth + elevation, and relative SE(3) trained on camera rings or free poses.
Scenes with more objects(Crowded Scenes).
PORT synthesizes a target view from a single input image containing a larger number of objects. Color-coded keypoints and segmentation masks show object correspondence across views.
Crowded scenes with visually similar objects. PORT synthesizes a target view from a single input image. Color-coded keypoints and segmentation masks show object correspondence across views.
Novel-view synthesis with textured backgrounds. PORT synthesizes a target view of a scene with a textured background from a single input image.
Cross-view transport with relative SE(3) conditioning in the original PandaPush camera setup. PORT transports object representations from the side camera to the front camera, which differ in height and distance, demonstrating accurate object transport under full relative camera transformations.
Object-centric transport across viewpoints with relative SE(3) conditioning. PORT predicts target-view object representations under a different camera viewpoint while preserving object correspondence, demonstrating that the transport mechanism extends beyond azimuth-only conditioning.
Abstract
We introduce Particle-based Object Representation Transport (PORT), a self-supervised object-centric model that directly transports object-centric latent representations extracted from a single source image to target viewpoints according to relative camera pose. Rather than merely generating novel-view images, PORT directly transports each object latent to predict its position, scale, depth, and appearance under different horizontal camera viewpoints. This enables PORT to construct viewpoint-conditioned multi-view object representations from a single image while preserving object correspondence across views, and to reason about the underlying 3D geometry by jointly leveraging object states across multiple viewpoints. We further provide object-level evaluation metrics and a dataset protocol for quantitatively assessing the accuracy of cross-view latent transport. We then demonstrate the utility of the resulting multi-view object representations in Gaussian Splatting and robotic manipulation, showing their applicability to both 3D scene representation and downstream tasks.
Our code will be publicly available.
1 Source Image → Transported Object Latents → Target-View Decomposition
POS: Object positions · SCALE: Object scales · SEG: Object masks · FG: Foreground · BG: Background
CLEVR3D
TP-OBJ3D
TP-GROCERY
Real World Scene
PandaPush
1 Source Image → Transported 2D Positions → 3D Geometry & Depth
CLEVR3D
TP-OBJ3D
Real World Scene
TP-GROCERY
1 Real View + PORT-Imagined Views → Policy Learning
PandaPush
PandaPush
1 Source Image → PORT-generated Views → 3D Gaussian Scene
TP-OBJ3D
TP-GROCERY
TP-GROCERY
Real World Scene
Original source image → Edited source image → Rotating Edited Views
TP-OBJ3D
TP-GROCERY
Original source image → Edited source image → Rotating Edited Views → Gaussian Splatting
TP-OBJ3D
TP-GROCERY