Introduction

Early on in development of our own deferred shader architecture we set about implementing a dynamic 'local' lighting solution for VFX, that provided lighting for explosions, headlights and street lights.

We had already implemented a global lighting pass - a full screen pass that shades the directional lit view of the scene and combines with deferred shadowing.

GPU local lighting

Our first local lighting implementation was based upon common techniques - rasterize light volume geometry on the GPU (e.g. point light spheres ) and calculate the lighting values per pixel in a lighting fragment shader.

The local light shading takes place in screen space; we shade the screen pixels that are covered by the light volume.

We also intersect test the light volumes with the scene using the depth and stencil so that we shade the areas only where the light volume intersects scene geometry

While shading each pixel we shoot a ray down the camera frustum and sample the Depth buffer, this reconstructs the world position of the surface under that pixel, information we need to attenuate the light. We also sample the Normal buffer to determine whether the surface is facing the light.

The resulting light value is then combined with GBuffer Albedo and accumulated into the Global Light Pass buffer, where the final lit image of the scene is generated.

We found that the ps3 (in comparison to the xbox 360) quickly became fill rate bound - large explosions were very expensive.

We decided to move the GPU local lighting to the SPU processors.

SPU local lighting

We needed the SPUs to generate local lighting in parallel with the GPU doing other work, like shadow cascades.

To calculate the local light values for a single screen space pixel we need the GBuffer Normal and the Depth buffers. We also need access to data that describes the lights covering that screen pixel; for example light position, color, type.

The ps3 has 8 SPU units, each with access to 256kb of fast RAM.

With a screen resolution of 1280x720 pixels, the 32bit GBuffer Normal and Depth buffers are 3.5MB each - clearly too much to fit on a single SPU and process in one job.

Tiling the screen

We split the screen into 64x64 pixel tiles, which happened to be the size of one ps3 main memory frame buffer tile.

We fed the SPUs with a queue of jobs, each job shaded a 64x64 pixel tile.

A screen resolution of 1280x720 would require 20x12 tiles or 240 SPU jobs.

SPU jobs

The SPU lighting takes place after the GBuffer had finished rendering - the GPU calls back to the PPU and this triggers the job queue to send the jobs to the SPUs.

With each SPU job and we send the tile's GBuffer Normal and Depth buffer to the SPU in a DMA call. We also send the list of lights touching each tile, classified in screen space.

Each SPU job executes and shades the light values for its 64x64 pixel tile, and then DMAs the result to the equivalent tile region in a dedicated Local Light buffer.

While these jobs are executing, the GPU goes on to render the shadow cascade, after which it sits on a command buffer JTS waiting for a signal that the SPU jobs have finished.

The final SPU job clears the JTS (wait) in the GPU command buffer and the GPU goes on to render the Global Light Pass, sampling the Local Light buffer.

Light classification per tile

Each SPU job contains the list of lights touching each tile, classified in screen space.

To classify, we built the list of lights visible to the camera frustum and depth sorted it, so that the nearest lights took priority in each tile.

We then found the screen space bounds of each light, and stepped over this to determine which tiles the light intersected. Each tile was tagged with the lights that touched it.

We also set a maximum limit of 16 lights per tile, this clamped the cost of shading.

SPU intrinsic code

Each job executes SPU code which shades the tile. The SPU executable code is written using intrinsic instructions, this gives optimal performance. We shade 4 lights at a time, to take advantage of SIMD parallelization.