Seeing without Pixels: Perception from Camera Trajectories

Options:

A. C holds the orange hold with his right hand.

B. C rises towards the bouldering wall.

C. C observes the bouldering wall.

D. C touches the yellow hold with his left hand.

E. C lands on the mat.

CamFormer prediction (camera trajectory as input): E ✅

Video baseline prediction (ego video as input): D ❌

A. C moves her left foot to the right, taps the floor and steps to the left with her left foot.

B. C raises her left foot behind her and slightly raises her right leg on her side.

C. C is stepping in place from side to side with her left and right legs alternately.

D. C bends both of her legs alternately.

E. C walks backwards with her left and right leg.

CamFormer prediction (camera trajectory as input): C ✅

Video baseline prediction (ego video as input): B ❌

A. A finger pointing at a cutting board with vegetables on it.

B. A man is wearing a cap and talking to the camera in front of a backdrop of mountains.

C. There is a woman standing in front of a garden filled with vegetables, she is smiling and holding a camera.

D. A veterinarian is using a syringe to draw blood from a newborn puppy.

E. A man is hiking in a forest and taking a selfie with his phone

CamFormer prediction (camera trajectory as input): A ✅