Figure 5: Video Predictions in Open-ended Environments (human hands and robots)
Approach the AirPod in front of the bottle.
Place the leftmost bottle in front of the cup.
Use the scissors to cut the paper.
Close the screen of the MacBook.
Grasp a tissue and wipe the screen.
Place the spoon on the iPad.
Pull out the gas nozzle.
Remove the lid and place it on the table.
Grasp the sponge and clean the whiteboard.
Place the electronic device into the box.
Place the green block on the controller.
Pour water into the blue bowl.
Fold the towel from right to left.
Open the upper drawer.
Sweep the block into the dustbin.
Grasp the blue towel and wipe the plate.
Figure 7: Comparisons with Wan-14B/Cosmos3-64B
Same Initial observation 1
Wan-2.1-I2V-14B
Cosmos3-Super-64B
Place the second bottle from the left in front of the leftmost bottle.
VPP2 (ours)
Same Initial observation 1
Wan-2.1-I2V-14B
Cosmos3-Super-64B
Place the leftmost bottle in front of the cup
VPP2 (ours)
Same Initial observation 1
Wan-2.1-I2V-14B
Cosmos3-Super-64B
Place the cup in front of the leftmost cup.
VPP2 (ours)
Same Initial observation 2
Wan-2.1-I2V-14B
Cosmos3-Super-64B
Stack the red paper cup on the leftmost black cup.
VPP2 (ours)
Same Initial observation 2
Wan-2.1-I2V-14B
Cosmos3-Super-64B
Stack the red paper cup on the second black cup from the left.
VPP2 (ours)
Demonstration of our self-collected data and thier processed caption
Aloha
Dex-hand
Ego