Check our final result live on your browser, in ShaderToy!
Final Report
Raine Koizumi, Arav Misra, Sophia Sunkin, Arjun Palkhade
Abstract
Ray marching is a technique for rendering that is essentially an accelerated version of ray tracing. In ray marching, rays are iteratively traversed by step sizes equal to the minimum distance between a given point on a ray to the nearest surface, calculated by sphere tracing. This means we can create much more complex scenes and geometry without increasing the render speeds. We modeled geometry on the scene using signed distance functions (SDFs) instead of meshes, meaning that objects like trees and terrain are all math equations. We have implemented Ray Marching in GLSL on Shadertoy and used this to render primitive shapes, as well as a procedurally generated complex scene in real time. In our final scene, we have created procedural terrain with animated trees, water, clouds, and sun, as well as implementing effects such as reflections on water, vignette, color post processing (gamma correction, contrast), shadows, phong shading, anti-aliasing, and fog.
Technical Aproach
We started with a blank GLSL file on Shadertoy, with no template code other than signed distance functions. Everything we produced visually and go on to describe below was the result of our own code, given for our terrain that we had no starter functionality. We started with the basic ray marching algorithm of primitive shapes, then calculating normals, shadows, and lighting, then utilizing operations such as union, difference, and displacement, then implementing materials, and finally sculpting the entire world along with effects such as sky and water.
Ray marching works by casting rays out from an origin point and traversing by the distance to the nearest surface, calculated by a sphere radius. Once a ray approaches a surface, the radius will grow smaller and smaller until it reaches a determined surface distance offset.
Ray marching code
Basic Operations
Signed distance functions are the basis of creating objects in ray marching. A signed distance function takes in the position of the marching ray as an argument, and returns the distance to the surface of that object.
Here is a simple example using a sphere:
The input p is a point on the ray being marched (and can be translated by another vector to change the sphere position), and s is the sphere radius.
In order to layer multiple objects in a scene, the union of these objects is taken by returning the minimum value of all objects’ signed distance functions. Intuitively, the object that will be rendered is closest to the ray’s origin, and thus the minimum distance. There are signed distance functions for a plethora of objects, including but not limited to planes, tori, cylinders, and cubes as pictured below.
There are a few other basic operators we implemented – pictured on the left in the image below, is a smooth union, computed by interpolating between the minimum of two objects. In the middle is the intersect operation, which takes the maximum between two objects. Lastly, boolean subtraction creates a “cutout” effect by taking the minimum between one object and another negated object.
Terrain
Initial test of noise functions during our checkpoint
We generated terrain using two noise functions: one for overall landscape and another for smaller details. Since the primitive that the terrain is made of is a simple plane, the signed distance field is simply the position minus the height of the object. In this case, our signed distance field would look like p - groundHeight. Here is what groundHeight looks like with one and both noise values, using equation: groundHeight = noise(p.xz * 0.01+124.1234) * 80. + noise(p.xz * 0.1) * 5.
noise1+noise2
(dark spots are not artifacts. they are shadows)
noise1
noise2
For the material, we used the same noise function as the smaller detail one to generate patches of grass, mixing between brown and green colors.
Trees
Trees were the most technically challenging part of our project. In order to model a forest, we needed to create an infinitely repeating random collection of tree objects. Thanks to a quirk of ray marching, calculating an infinite amount of trees is actually the same computation as calculating a single tree!
Here is how this work.
Repetition was attained by first creating a simple grid of objects in our scene. To do this, we used the fract function, which takes in a float and returns only its decimal part. This is useful because we can use this to determine the distance from a position onto its closest point on a grid.
Here is an example of how fract provides the distance to the closest grid point:
With some simple transformations, we can scale this to any grid size, as well as change it to get the distance to the center of each grid rather than in the corner.
Now, the ray which originally had its position variable P, will instead have a variable Q that transforms P into a value between 0 and 1 that constantly repeats. If we generate a signed distance field for an object (such as a sphere) centered at vec2(0.5, 0.5), this object will appear at every 1 distance apart.
Basic grid generated using fract
Code for above image
Next, we implemented two things: random jittering of the tree positions on the grid, and offsetting the height to be the same as the terrain. This part was relatively simple, we can use a random function and pass in the floored coordinate as input.
Added jitter to grid as well as terrain offet
Code for above image
We modeled the shape of a tree by generating three overlapping cone objects, applying noise to each, and taking the union of them to construct a cohesive ‘foliage’ appearance. Next we added an inner ‘cone’ which serves to provide visual volume to the trees, as well as having the effect of color variation within the foliage itself.
Image of trees built using cones and noise
Then, we modified the material to have a random color between dark and light green using noise.
Lastly, we animated our trees using sine waves, to give an illusion of wind!
Sky
The two components of rendering the sky were creating clouds and the sun. Starting with clouds, we used fractional brownian motion (FBM) to generate noise. Brownian motion is the random change in position of particles suspended in liquid or gas, and FBM is an extension of this function with an extra parameter that provides a property of persistence such that movement in one direction will continue to tend in that direction. For the purpose of creating realistic clouds, FBM serves as a noise function with smooth transitions between sky and clouds to create a hazy look. The final FBM function starts out with a basic noise function and iteratively adds more detailed noise at each step by increasing the frequency by a factor of two and layering that on top of the previously calculated noise.
Fractional Brownian Motion code
For the sun, we used a technique similar to view direction dependent specular reflection of the blinn-phong shading model. We first defined a point in space where the sun is centered at, used that to determine the direction of the light ray, and took the dot product of that light direction and the ray being traversed. The larger that dot product value is, the closer the traversed ray is to the center of the sun, and the brighter the output pixel is.
Sun without sky or environment
Water & Reflections
To render bodies of water, we create a static plane of blue and applied time-dependent noise to create the effect of small undulations, aka waves. To create the reflection of the trees and sky, we used the normal vector of the water and the direction of the viewing ray to calculate the direction of the reflected ray using the reflection function:
output ray direction = ray_input - 2 * N (ray_input * N)
This output ray’s resultant color is either the trees’ reflection or determined by the sky generation function.
Water reflections without land
Water reflections with land
Water reflections with land and trees
Fog
The goal of fog was to create a greater sense of depth in our scene. To simulate fog, we use our ray distance in an exponent so that further objects will be colored more like the fog color. The fog equation is:
1.0 - exp(-fog_intensity * distance * distance)
No fog
Blue fog with intensity 0.0000005
White fog with intensity 0.000003
Yellow fog with intensity 0.000000
Vignette and contrast
To give a more cinematic feel, vignette was applied, along with increasing the contrast of the scene. Our equation was col *= 0.5 + 0.9*pow(16.0*vig_uv.x*vig_uv.y*(1.0-vig_uv.x)*(1.0-vig_uv.y), 0.5 );
This equation essentially calculates how close the uv coordinate is to the edge. It had an added bonus of increasing the contrast in our scene.
Here is a before and after:
Challenges
Artifacts were common throughout our project, stemming from one issue: high frequency displacement mapping. This issue comes from the simple fact that signed distance functions are inaccurate when certain displacement operations are made, because rays might step over objects.
Here are a couple of cases we encountered, and how we solved the issue:
Issue 1: Overstepping into terrain
Occasionally, the ray marching algorithm would step into the terrain. To fix this,we added an absolute value to our ray marching end condition, so that our code will backtrack if it steps into an object (originally it would stop immediately if it stepped into an object).
Old code
New code
Issue 2: stepping through objects
For higher frequency / more detailed objects, such as trees, a common issue was overstepping. To fix this issue, we simply step forward a fraction of the amount that the signed distance function tells us to step forward. That way, we have a higher probability of actually hitting the object, but at the expense of computing time and requiring more steps to calculate a scene
Old code
New code
Issue 3: Performance
Higher quality renders result in lower performance. Since this is a live demo, we wanted to get our frame rate at 60FPS at all times.
There were many things that contributed to the render speed of our scene. Here are some to count
MAX_STEPS, MAX_DIST, SURF_DIST parameters for general ray marching algorithm
Number of objects to calculate that require distance functions. This only becomes an issue when a single SDF (signed distance function) is complex and takes up most of the scene. Most calculations tend to be fast, however, so there were no retrospective optimizations required. One thing to note is that it is practically free to draw many trees vs drawing only one tree. This is a quirk of ray marching, because instead of calculating a thousand trees per ray (a hypothetical tree count in a traditional ray casting algorithm), ray marching only has to calculate one tree per march iteration (up to MAX_STEPS per full march), yet still produces a thousand trees. In fact, it will calculate an infinite amount of trees!
Water reflection requires an additional call to the ray march function. In a scene directly pointed towards a large surface of water, with increased render parameters for quality, we saw a performance increase of 40 to 45 FPS when switching from reflections ON to reflections OFF.
Anti-aliasing was implemented by simple up sampling, then averaging. This quadruples the entire computation power required, and thus we did not use it in our final render. However, you can see the effects (visual & performance) in images below.
Default 60FPS
MAX_STEPS 300
MAX_DIST 500.
SURF_DIST 0.12
We chose these values to have as much visual quality that doesn't lower the FPS below 60.
Lower max steps 60FPS
MAX_STEPS 100
MAX_DIST 500.
SURF_DIST 0.12
increasing max steps does not improve the visual quality at all, nor does it impact frame rate as much when increasing from base. Therefore, here we show the effects of decreasing the max steps. As you can see, the ray marching algorithm has a harder time of determining object hits in the distance.
Increased MAX_DIST and MAX_STEPS 55FPS
MAX_STEPS 600
MAX_DIST 1600.
SURF_DIST 0.12
Here, we increase the max distance of render. You can see more mountains further back. As demonstrated with the previous test to the left, to determine distant object hits, we would also have to increase the max steps as well.
Increased SURF_DIST 35FPS
MAX_STEPS 300
MAX_DIST 500.
SURF_DIST 0.01
This parameter had the largest impact on frame rate. SURF_DIST determines how close an object needs to be to determine object hit. This means that objects will appear higher in detail, and at the same time also require more iterations to render. Because we are rendering a big scene where smaller details (like the trees) matter less, it's actually visually better to keep this value somewhat high, so everything blends in a little better. (less aliasing we have to deal with!)
Anti-aliasing disabled
60 FPS
Flickering from trees, sharper edges
Anti-aliasing enabled
17FPS
Large visual performance increase at the expense of a much higher render time. Less flickering from trees. 4 samples per pixel.
Results
Here's the GIF version!
References
"Painting Landscape with Maths" by Inigo Qilez. Our original inspiration for this project. https://youtu.be/BFld4EBO2RE?si=FSmlXUn_oBXYkdLb
Fractional Brownian Motion, for clouds: https://iquilezles.org/articles/fbm/
Signed Distance Functions: https://iquilezles.org/articles/distfunctions/
GPU gems 2, per-pixel displacement mapping with distance functions: https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-8-pixel-displacement-mapping-distance-functions
"Ray Marching for Dummies!" by The Art of Code. Basis for our original ray marching template. https://youtu.be/PGtv-dBi2wE?si=XFLx_o4_c5-q0stR
Light reflections, for the water: https://paulbourke.net/geometry/reflected/
Contributions
Each team member created a basic ray marching scene from scratch to learn the basics during week 1 and 2. For weeks 3 and 4, we each picked a certain aspect of the forest terrain to model.
Raine Koizumi: Terrain generation, lighting, fog, and fixing visual artifacts
Arav Misra: Water & reflections
Sophia Sunkin: Sky/Clouds & sun
Arjun Palkhade: Tree modeling & animation, vignette