TPA-Net: Generate A Dataset for Text to Physics-based Animation
An autonomous data generation technique and a dataset to narrow the gap with a large number of multi-modal, 3D Text-to-Video/Simulation data
Examples
DROP-from
Language Caption Examples
Example 1 (SVO pattern)
Sentence-template: A rough, super large and blue treasure chest that is fluid and mediumweight is dropping from a container.
Sentence-synonym: A crude, super-sized, and blue treasure box in the surface fluids and medium weights is falling from the container.
Sentence-translation: A super large, blueish, and uncut treasure chest which is mediumweight and liquid, is knocking off from a container.
Example 2 (OVS pattern)
Sentence-template: A container which is even-surface, yellow and elastic is devolved by a blue and fluid treasure chest which is mediumweight.
Sentence-synonym: A container that is elastic, even-surface and sensationalistic is devolved by a bluish and liquid treasure chest which is mediumweight.
Sentence-translation: Ground surface, yellow and flexible containers are transferred from medium blue and fluid treasure boxes.
Example 3 (SV pattern)
Sentence-template: A super large, glossy and mediumweight treasure chest that is fluid drops.
Sentence-synonym: A super large, glossy, and mediumweight treasure chest that is liquid unloads.
Sentence-translation: A very large, smooth, fluid, and medium-weight treasure box falls down.
DROP-to
Language Caption Examples
Example 1 (SVO pattern)
Sentence-template: A yellow, sand and small flying-saucer which is even-surface is dropping to a receptacle that is medium-sized, glossy and mediumweight.
Sentence-synonym: A yellow, sand and small flying-saucer which is even-surface is dropping to a receptacle that is medium-sized, glossy and mediumweight.
Sentence-translation: Yellow, sand and small sardines (plane) are dropping into a medium, smooth and medium-weight receptacle.
Example 2 (SVP pattern)
Sentence-template: A yellow flying-saucer on the ground dropped.
Sentence-synonym: An icteric flying-saucer on the ground, devolved
Sentence-translation: A yellow sarcophagus down on the ground.
Example 3 (SVP pattern)
Sentence-template: An even-surface flying-saucer which is sand and matte is dropping to a red and elastic receptacle on the ground.
Sentence-synonym: An even-surface flying saucer that is sand and matte, is neglecting to a ruby and elastic receptacle on the ground.
Sentence-translation: The surface-flying sand and ground mats are dropping on a red elastic receptacle.
PUSH-from
Highlight Labels
Simulator: IPC
# target-of-interest = 2; # collision-object = 1
Material (Young's Modulus):
target_0 = elastic (soft), target_1 =elastic (hard)
collision_0 = rigid (collision objects are treated as rigid unmovable boundary condition)
Render color: target_0 = blue, target_1 = green, collision_0: green
Dynamic:
Language Caption Examples
Example 1 (SPV pattern)
Sentence-template: A mediumweight and green venus that is large on the ground is pushing from a puzzle.
Sentence-synonym: A green and mediumweight venus that is big on the ground, is driving from a puzzler
Sentence-translation: A giant medium and green fang on the ground is pushing from a puzzle.
Example 2 (OVSP pattern)
Sentence-template: A makerbot which is super large on the ground is pushed is pushed by a green, matte and large venus which is even-surface and mediumweight on the ground.
Sentence-synonym: A makerbot which is super large on the ground is pushed forrad by a with child , light-green and mat venus which is mediumweight and even-surface on the ground.
Sentence-translation: The mega-manufacturers on the surface are pushed by green maws and huge venoms on the ground, even on the surface and medium weights of green maws and huge venoms.
Example 3 (OVS pattern)
Sentence-template: A matte and even-surface makerbot that is super large is pushed fast by a venus which is even-surface on the ground.
Sentence-synonym: A matted and even-surface makerbot that is super large is forced fast by a venus that is even-surface on the ground.
Sentence-translation: The super-large stones, even surface makers, are quickly pushed by even snakeheads on the ground.
PUSH-to
Language Caption Examples
Example 1 (OVSP pattern)
Sentence-template: A large and rough linklings that is matte is pushed right fast by a medium-sized and even-surface snowmonster which is elastic on the ground.
Sentence-synonym: A harsh and declamatory linklings that is matte is pushed right fast by a medium-sized and even-surface snowmonster which is elastic on the ground.
Sentence-translation: Large and rough connectors are rapidly driven by elastic mid-sized or even surface snow objects on the ground.
Example 2 (SVO pattern)
Sentence-template: An even-surface and medium-sized snowmonster on the ground is pushing a linklings which is matte, white and large to a blue makerbot.
Sentence-synonym: A moderate-sized and even-surface snowmonster on the ground, is bearing on a linklings that is whitened, magnanimous and flat to a bluish makerbot.
Sentence-translation: A snowman of both plane size and medium size on the ground is pushing a mated, white and large connector towards a blue maker robot.
Example 3 (SV pattern)
Sentence-template: A mediumweight, glossy and medium-sized snowmonster pushes.
Sentence-synonym: A mediumweight, specious and medium-sized snowmonster, pushes.
Sentence-translation: Medium weight, smooth and medium-scale snowman thrust.
Highlight Labels
Simulator: IPC
# target-of-interest = 2; # collision-object = 1
Material (Young's Modulus):
target_0 = elastic (moderate-hardness), target_1 =elastic (soft)
collision_0 = rigid (collision objects are treated as rigid unmovable boundary condition)
Render color: target_0 = white, target_1 = green, collision_0: blue
Dynamic:
STRIKE
Language Caption Examples
Example 1 (OVS pattern)
Sentence-template: A super large mario which is smooth and snow on the ground is affected fast horizonly by a totoro which is elastic , mediumweight and smooth.
Sentence-synonym: A super large mario that is suave and snow on the ground is affected fast horizonly by a totoro which is mediumweight , pliable and smooth.
Sentence-translation: Super-large Mario smooth and snow is affected by the elastic, medium and smooth Totoro fast horizon.
Example 2 (SVO pattern)
Sentence-template: A mediumweight, elastic and yellow totoro strikes to a super large mario which is smooth, lime and light.
Sentence-synonym: An elastic, mediumweight and xanthous totoro, expunges to a super large mario which is unruffled, idle and lime.
Sentence-translation: Medium, elastic and yellow Totoro hit a super-large horse, smooth, smooth, lime and bright.
Example 3 (SVO pattern)
Sentence-template: A totoro which is smooth and super large on the ground is striking to a smooth mario which is lime and super large.
Sentence-synonym: A totoro that is suave and super large on the ground, is assuming to a suave mario which is lime and super large.
Sentence-translation: A smooth and super-large-grounded Totoro stormed a smooth Mario, i.e. lime and super-large.
More demos are on the way ...
We are improving the dataset from the following perspectives:
More colorful and accurate captions
More sentence patterns
More clever sentence sampler
More dynamic models
More verbs
More object relationships
More integrated physical simulator other than MPM and IPC