Check our final result live on your browser, in ShaderToy! 

https://www.shadertoy.com/view/XfdXz2

Final Report

Raine Koizumi, Arav Misra, Sophia Sunkin, Arjun Palkhade

Abstract

Ray marching is a technique for rendering that is essentially an accelerated version of ray tracing. In ray marching, rays are iteratively traversed by step sizes equal to the minimum distance between a given point on a ray to the nearest surface, calculated by sphere tracing. This means we can create much more complex scenes and geometry without increasing the render speeds. We modeled geometry on the scene using signed distance functions (SDFs) instead of meshes, meaning that objects like trees and terrain are all math equations. We have implemented Ray Marching in GLSL on Shadertoy and used this to render primitive shapes, as well as a procedurally generated complex scene in real time. In our final scene, we have created procedural terrain with animated trees, water, clouds, and sun, as well as implementing effects such as reflections on water, vignette, color post processing (gamma correction, contrast), shadows, phong shading, anti-aliasing, and fog.

Technical Aproach

We started with a blank GLSL file on Shadertoy, with no template code other than signed distance functions. Everything we produced visually and go on to describe below was the result of our own code, given for our terrain that we had no starter functionality. We started with the basic ray marching algorithm of primitive shapes, then calculating normals, shadows, and lighting, then utilizing operations such as union, difference, and displacement, then implementing materials, and finally sculpting the entire world along with effects such as sky and water. 


Ray marching works by casting rays out from an origin point and traversing by the distance to the nearest surface, calculated by a sphere radius. Once a ray approaches a surface, the radius will grow smaller and smaller until it reaches a determined surface distance offset. 

Ray marching code

Basic Operations

Signed distance functions are the basis of creating objects in ray marching. A signed distance function takes in the position of the marching ray as an argument, and returns the distance to the surface of that object. 


Here is a simple example using a sphere:

The input p is a point on the ray being marched (and can be translated by another vector to change the sphere position), and s is the sphere radius.


In order to layer multiple objects in a scene, the union of these objects is taken by returning the minimum value of all objects’ signed distance functions. Intuitively, the object that will be rendered is closest to the ray’s origin, and thus the minimum distance. There are signed distance functions for a plethora of objects, including but not limited to planes, tori, cylinders, and cubes as pictured below. 



There are a few other basic operators we implemented – pictured on the left in the image below, is a smooth union, computed by interpolating between the minimum of two objects. In the middle is the intersect operation, which takes the maximum between two objects. Lastly, boolean subtraction creates a “cutout” effect by taking the minimum between one object and another negated object.

Terrain

Initial test of noise functions during our checkpoint

We generated terrain using two noise functions: one for overall landscape and another for smaller details. Since the primitive that the terrain is made of is a simple plane, the signed distance field is simply the position minus the height of the object. In this case, our signed distance field would look like p - groundHeight. Here is what groundHeight looks like with one and both noise values, using equation: groundHeight = noise(p.xz * 0.01+124.1234) * 80. + noise(p.xz * 0.1) * 5. 

noise1+noise2

(dark spots are not artifacts. they are shadows)

noise1


noise2

For the material, we used the same noise function as the smaller detail one to generate patches of grass, mixing between brown and green colors. 

Trees

Trees were the most technically challenging part of our project. In order to model a forest, we needed to create an infinitely repeating random collection of tree objects. Thanks to a quirk of ray marching, calculating an infinite amount of trees is actually the same computation as calculating a single tree! 


Here is how this work.


Repetition was attained by first creating a simple grid of objects in our scene. To do this, we used the fract function, which takes in a float and returns only its decimal part. This is useful because we can use this to determine the distance from a position onto its closest point on a grid. 


Here is an example of how fract provides the distance to the closest grid point:

With some simple transformations, we can scale this to any grid size, as well as change it to get the distance to the center of each grid rather than in the corner. 


Now, the ray which originally had its position variable P, will instead have a variable Q that transforms P into a value between 0 and 1 that constantly repeats. If we generate a signed distance field for an object (such as a sphere) centered at vec2(0.5, 0.5), this object will appear at every 1 distance apart. 

Basic grid generated using fract

Code for above image


Next, we implemented two things: random jittering of the tree positions on the grid, and offsetting the height to be the same as the terrain. This part was relatively simple, we can use a random function and pass in the floored coordinate as input.

Added jitter to grid as well as terrain offet

Code for above image


We modeled the shape of a tree by generating three overlapping cone objects, applying noise to each, and taking the union of them to construct a cohesive ‘foliage’ appearance. Next we added an inner ‘cone’ which serves to provide visual volume to the trees, as well as having the effect of color variation within the foliage itself. 

Image of trees built using cones and noise


Then, we modified the material to have a random color between dark and light green using noise.

Lastly, we animated our trees using sine waves, to give an illusion of wind!

Sky

The two components of rendering the sky were creating clouds and the sun. Starting with clouds, we used fractional brownian motion (FBM) to generate noise. Brownian motion is the random change in position of particles suspended in liquid or gas, and FBM is an extension of this function with an extra parameter that provides a property of persistence such that movement in one direction will continue to tend in that direction. For the purpose of creating realistic clouds, FBM serves as a noise function with smooth transitions between sky and clouds to create a hazy look. The final FBM function starts out with a basic noise function and iteratively adds more detailed noise at each step by increasing the frequency by a factor of two and layering that on top of the previously calculated noise.

Fractional Brownian Motion code


For the sun, we used a technique similar to view direction dependent specular reflection of the blinn-phong shading model. We first defined a point in space where the sun is centered at, used that to determine the direction of the light ray, and took the dot product of that light direction and the ray being traversed. The larger that dot product value is, the closer the traversed ray is to the center of the sun, and the brighter the output pixel is. 

Sun without sky or environment

Water & Reflections

To render bodies of water, we create a static plane of blue and applied time-dependent noise to create the effect of small undulations, aka waves. To create the reflection of the trees and sky, we used the normal vector of the water and the direction of the viewing ray to calculate the direction of the reflected ray using the reflection function:

output ray direction = ray_input - 2 * N (ray_input * N)

This output ray’s resultant color is either the trees’ reflection or determined by the sky generation function.


Water reflections without land


Water reflections with land


Water reflections with land and trees

Fog

The goal of fog was to create a greater sense of depth in our scene. To simulate fog, we use our ray distance in an exponent so that further objects will be colored more like the fog color. The fog equation is:

1.0 - exp(-fog_intensity * distance * distance)


No fog


Blue fog with intensity 0.0000005


White fog with intensity 0.000003


Yellow fog with intensity 0.000000

Vignette and contrast

To give a more cinematic feel, vignette was applied, along with increasing the contrast of the scene. Our equation was col *= 0.5 + 0.9*pow(16.0*vig_uv.x*vig_uv.y*(1.0-vig_uv.x)*(1.0-vig_uv.y), 0.5 );

This equation essentially calculates how close the uv coordinate is to the edge. It had an added bonus of increasing the contrast in our scene. 


Here is a before and after:

Challenges

Artifacts were common throughout our project, stemming from one issue: high frequency displacement mapping. This issue comes from the simple fact that signed distance functions are inaccurate when certain displacement operations are made, because rays might step over objects. 

Here are a couple of cases we encountered, and how we solved the issue:


Issue 1: Overstepping into terrain

Occasionally, the ray marching algorithm would step into the terrain. To fix this,we added an absolute value to our ray marching end condition, so that our code will backtrack if it steps into an object (originally it would stop immediately if it stepped into an object). 

Old code

New code

Issue 2: stepping through objects

For higher frequency / more detailed objects, such as trees, a common issue was overstepping. To fix this issue, we simply step forward a fraction of the amount that the signed distance function tells us to step forward. That way, we have a higher probability of actually hitting the object, but at the expense of computing time and requiring more steps to calculate a scene

Old code

New code

Issue 3: Performance

Higher quality renders result in lower performance. Since this is a live demo, we wanted to get our frame rate at 60FPS at all times. 

There were many things that contributed to the render speed of our scene. Here are some to count


Default 60FPS

MAX_STEPS 300

MAX_DIST 500.

SURF_DIST 0.12

We chose these values to have as much visual quality that doesn't lower the FPS below 60.

Lower max steps 60FPS

MAX_STEPS 100

MAX_DIST 500.

SURF_DIST 0.12

increasing max steps does not improve the visual quality at all, nor does it impact frame rate as much when increasing from base. Therefore, here we show the effects of decreasing the max steps. As you can see, the ray marching algorithm has a harder time of determining object hits in the distance.

Increased MAX_DIST and MAX_STEPS 55FPS

MAX_STEPS 600

MAX_DIST 1600.

SURF_DIST 0.12

Here, we increase the max distance of render. You can see more mountains further back. As demonstrated with the previous test to the left, to determine distant object hits, we would also have to increase the max steps as well.

Increased SURF_DIST 35FPS

MAX_STEPS 300

MAX_DIST 500.

SURF_DIST 0.01

This parameter had the largest impact on frame rate. SURF_DIST determines how close an object needs to be to determine object hit. This means that objects will appear higher in detail, and at the same time also require more iterations to render. Because we are rendering a big scene where smaller details (like the trees) matter less, it's actually visually better to keep this value somewhat high, so everything blends in a little better. (less aliasing we have to deal with!)

Anti-aliasing disabled

60 FPS

Flickering from trees, sharper edges

Anti-aliasing enabled

17FPS

Large visual performance increase at the expense of a much higher render time. Less flickering from trees. 4 samples per pixel.

Results

Here is an image of our final render.

Shadertoy link: https://www.shadertoy.com/view/XfdXz2

Here's the GIF version!

References

"Painting Landscape with Maths" by Inigo Qilez. Our original inspiration for this project. https://youtu.be/BFld4EBO2RE?si=FSmlXUn_oBXYkdLb
Fractional Brownian Motion, for clouds: https://iquilezles.org/articles/fbm/

Signed Distance Functions: https://iquilezles.org/articles/distfunctions/

GPU gems 2, per-pixel displacement mapping with distance functions: https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-8-pixel-displacement-mapping-distance-functions

"Ray Marching for Dummies!" by The Art of Code. Basis for our original ray marching template. https://youtu.be/PGtv-dBi2wE?si=XFLx_o4_c5-q0stR

Light reflections, for the water: https://paulbourke.net/geometry/reflected/

Contributions

Each team member created a basic ray marching scene from scratch to learn the basics during week 1 and 2. For weeks 3 and 4, we each picked a certain aspect of the forest terrain to model. 


Raine Koizumi: Terrain generation, lighting, fog, and fixing visual artifacts

Arav Misra: Water & reflections

Sophia Sunkin: Sky/Clouds & sun

Arjun Palkhade: Tree modeling & animation, vignette