Billboarding

Filming Clones

We saw a third-party Unity build of an undersea world with hundreds of nettles, fish, coral and krill. At that time, we were working on a CPU heavy interactive 3D nettle and it was taking up 85% of the CPU. We needed a way to show way more content with the remaining 15%. An optimization technique was needed. Using virtual cameras and virtual screens to film and project multiple copies of the nettles at arm's distance from the viewer. Unity can show an additional 30 to 50 stereoscopic billboarded nettles while still maintaining 120 fps. Enough CPU was left over to show an additional 1500 2D fish using a single mesh, with one quad per fish.

How it works

We start with some CPU heavy interactive content that we want to feature. We also want 50 additional copies of the featured content that look and behave randomly.

Cameras: Cameras positions around the featured content are determined by comparing the viewer transform with the billboard parent transforms.

Billboards: Billboards are quads that always face the viewer. AI controls the transform of the billboard parent.

Render textures: Unity's render textures work well to link cameras to quads.

Stereoscopic 3D: Two sets of camera / billboard are used for each content copy, one set for each eye.

Fog: Alpha levels of each billboard is adjusted for better depth-perception. Objects fade with distance.

Zones

To optimize CPU usage, interactive content exists only within reach of the viewer in 3D. Nearby content is stereoscopic, while far away content is 2D. Resolution of content and refresh rate is higher for nearer content.

Zone 1: The zone within the viewer's arm's length contains the viewer and interactive content.

Zone 2: The zone beyond the viewer's arm's length contains stereoscopic 3D billboards.

Zone 3: The next zone beyond contains 2D billboards and mesh-based 2D fish (quads).

Zone 4: The last zone contains background content.

Goal: To display hundreds of nettles just out of reach (not interactive) with remaining CPU.

Issue: One fully-loaded interactive 3d nettle with physics takes up most of the cpu at 120 fps.

Method: Use multiple pairs of camera and display screen (billboard) to film the nettle from different angles using AI to position the billboards.

Details: resolution as low as 256 x 256 work for fish, 1024 x 1024 for nettles. Aspect ratios can be adjusted.

Randomize: Speed, scale and mirroring maintain the appearance of unique object behavior.

Limits: The number of cameras, the number of game objects and the resolution affect the frame rate.

Billboarding vs Mesh Control: Billboarding allows individual camera streams for each billboard, while mesh quads display the same stream/texture, with UV offsets.

Proof of Concept: Team members can not tell the difference between billboarded content and original content in stereoscopic 3D.

Result: We are able to show up to 50 animated duplicates at 120 fps.

Stereoscopic 3D: Two cameras per viewer are used, one for each eye. The billboards display content above, synced with their respective cameras, and controlled by AI.

The top car is a billboard. The moving camera streams to the billboard.

Virtual camera transforms based on viewer position relative to billboard.

One of these is a 3D object, the other uses billboards.

Billboarding works with transparency with some variance.

Stereoscopic 3D: Each object is two billboards facing left and right eyes. Fish AI controls each object.

Page updated

Report abuse