Week07

Introduction to Shaders: From CPU to GPU

Agenda

From Programming Behavior to Programming Appearance
- Introduction to Shader
- CPU vs GPU
- Shader and GPU in XR
- Experiment with Shader by modifying properties and simple node connections.

From Programming Behavior to Programming Appearance

C# · GameObjects · Position · Velocity · Forces -> GPU · Shader · UV · Color · Mathematical Fields

In previous weeks, we used C# to describe gravity, N-body motion, Boids, and interactive behavior.
Those objects were already rendered by the GPU using shaders, even though we did not need to write shaders ourselves.
This week we begin to design our own visual rules.

Explicit Shapes

Implicit Field

The same three control points can produce separate circular objects or a connected silhouette.

“Can we create complex visual effects without creating complex geometry?”

What if we want to create flowing water, glowing trails, or liquid-like shapes that continuously merge and separate?

We could build more GameObjects or generate complex meshes, but do we really need to model every visual detail?

Make moving cubes with colors?

Shader

A shader is a program executed on the GPU that participates in rendering.

Shaders can process vertices and calculate how fragments of an image should appear.

In modern real-time graphics, they are how we specify many of the mathematical rules behind color, lighting, transparency, patterns, and certain geometry effects.

Core Rendering Concepts in Unity

Similar concepts are used in many 3D graphics applications, although names and implementations may differ.

GameObject + Transform: A GameObject organizes components in a scene, while its Transform defines its position, rotation, and scale.
Mesh: Defines geometry using vertices, triangles, and other attributes such as UV coordinates.
Renderer: A component responsible for rendering an object's geometry or other visual representation.
Material: Determines which Shader to use and provides properties such as colors, textures, and other parameters.
Texture: Image or data that a Shader can sample. Not all shaders require textures.
Shader: A program that runs on the GPU to calculate how geometry and pixels are rendered.

Shaders offer a different approach: we can describe what we want to see using mathematics.

Instead of calculating only the positions of objects, we can calculate the appearance of individual fragments on a surface.
GPUs are especially good at performing similar calculations across many fragments in parallel.

Vertex: A point used to define the geometry of a mesh. Each vertex has a position and may contain additional information such as UV coordinates, normals, and colors.
Fragment: A sample generated during rasterization when a triangle is projected onto the screen. Each fragment contains interpolated information, such as UV coordinates, which a fragment shader uses to calculate its appearance.
UV Coordinates: Two-dimensional coordinates (U, V) used to map positions on a 3D surface to a 2D space. They are commonly used for texture mapping and procedural shader calculations.

Vertices define the shape.
Fragments help determine how that shape appears on the screen.

Rasterization: The process of converting projected geometric primitives (such as triangles) into fragments that can be processed by a fragment shader.

Vertex Shader vs. Fragment Shader

Vertex Shader: Processes each vertex, primarily transforming its position. It can also modify vertex positions to deform geometry.
Fragment Shader: Processes fragments to calculate their appearance, including color, transparency, and surface properties.

Vertex vs Fragment Shaders: How the Graphics Pipeline Runs

Fractal: coined by the mathematician Benoit Mandelbrot in 1975

“A rough or fragmented geometric shape that can be split into parts, each of which is (at least approximately) a reduced-size copy of the whole.” The Fractal Geometry of Nature, Benoit Mandelbrot
https://natureofcode.com/fractals/
Fractals are characterized by having a fine structure at small scales (keep zooming in and you’ll continue to find fluctuations)

Think of a shader as a rule for drawing, rather than a new 3D object.

A constant-color shader can make a surface entirely red.
A pattern shader can make that surface striped, animated, or glowing without importing an image.
A vertex shader can change vertex positions so a surface ripples or bends.

Important: A shader is not the same thing as a Material.

A Material chooses a shader and supplies parameters to it. And a shader is not automatically a pixel shader: today's basic graphics shaders include both vertex and fragment stages. More specialized GPU programs, such as compute shaders, also exist.

* You were already using shaders. Add a default Cube in a Unity URP scene. Even before writing your own shader, its Material uses a prebuilt shader (for example, URP Lit) to determine how it is rendered.

Official introduction: Unity — Introduction to shaders.

Examples)

Shadertoy: https://www.shadertoy.com/playlist/featured

CPU vs GPU Comparison

C# on the CPU typically updated motion, interactions, and scene state. For example, Boids might use an Update() routine to change their positions. To render an image, the GPU runs graphics operations across many vertices and fragments. Thousands of fragments may apply a similar rule, each with different input coordinates. GPUs are designed for high-throughput parallel graphics computation.

The two processors cooperate:

C# / CPU question: Where is each Boid right now? What did the user do?
GPU / shader question: At this position on a rendered surface, what should the visual result be?

*ALU (Arithmetic Logic Unit): The hardware component that performs arithmetic and logical operations inside a processor.

The CPU can update a small set of positions while the GPU uses those positions to render a field across many fragments.

CPU: A few adults solving different, complicated problems.

GPU: Thousands of kids solving similar, simple problems at the same time.

Short Video: CPU vs GPU

Do not confuse “GPU is designed for parallel work” with “GPU is always faster.”

Complex shaders can be expensive. Resolution, overdraw, repeated calculations, memory transfer, and XR hardware all matter.

Example of Shader based graphics

Compute Shaders: Beyond Graphics

Compute shaders allow us to use the GPU for general-purpose computing (GPGPU), not just rendering graphics.

Unlike vertex and fragment shaders, compute shaders operate outside the traditional graphics rendering pipeline. They can perform massively parallel calculations on large amounts of data.

Common applications include particle simulations, fluid dynamics, physics, and image processing.

Why Are GPUs So Important for Modern AI?

Modern AI models perform enormous amounts of mathematical computation, particularly matrix multiplication.

Training and running neural networks involve repeatedly applying similar mathematical operations to large arrays of numbers called tensors.

GPUs are particularly effective at this because they can process many calculations in parallel. Modern GPUs also include specialized hardware, such as Tensor Cores, designed to accelerate matrix operations.

This is one of the main reasons GPUs have become essential for modern AI training and inference.

Think about the connection:

Fragment Shader: Performs similar mathematical operations across millions of fragments.
Compute Shader: Performs parallel calculations on large datasets.
AI Computing: Uses the same fundamental advantage of GPU parallelism to process enormous matrices.

The key idea: GPUs are not just for graphics anymore. They are powerful parallel computing engines.

Approx. Computation (MACs: (Multiply-Accumulate Operation)) of AI Models

ANN: 100k
DNN: 3 millions
CNN: 4 millions
Transformer: 11.2 billion
LLM (Small): 7 billion
LLM (Large): 70 billion

Why Does GPU Computing Matter in XR?

XR applications require more than just rendering 3D graphics. They must continuously respond to head movements, track user interactions, and sometimes process spatial information and AI models, all in real time.

Unlike traditional applications, XR must render separate views for both eyes while maintaining a consistently high frame rate.

At 90 FPS, each frame has a time budget of only 11.1 milliseconds.

GPUs make these experiences possible by processing large numbers of calculations in parallel.

However, GPU resources are limited, especially on standalone XR headsets.

The challenge is not just how much we can compute, but how much we can compute within each frame.

This is why understanding shaders, GPU computing, and performance optimization is essential for XR development.

Week01 - XR

Strict latency budgets require keeping motion-to-photon (MTP) from head movement to lit pixels extremely low, typically under ~20 ms for comfortable VR. This budget spans sensor fusion and pose prediction, app CPU/GPU render time, reprojection (ATW/ASW), display scan-out, and pixel response.

Q: Do XR headsets actually have GPUs?

A: Yes, Standalone XR headsets have integrated GPUs, just like smartphones.
For example, Meta Quest 3 uses a Snapdragon chip with a built-in Adreno GPU.
Unlike desktop graphics cards, mobile GPUs operate under strict power and thermal constraints.
That's why shader optimization matters even more in standalone XR.

Q: What if my laptop doesn't have a GPU?

A: Most laptops have an integrated GPU built into the processor, even without a dedicated graphics card.

You can still run shaders on an integrated GPU. However, complex shaders may run much slower than on a powerful dedicated GPU.

Creating Shaders in Unity

Starter Examples available. git pull and get the updates.

Color

Gradient

Circle

Unity gives us two main authoring approaches for graphics shaders:

Shader Graph: Connect nodes visually to describe calculations. This creates a shader program; it is not simply an image-editing tool.
(https://docs.unity3d.com/Packages/com.unity.shadergraph@17.1/manual/Getting-Started.html)
High-level shader language (HLSL): Author shader structure and GPU code directly. Useful when we need more custom control

We'll start with a single Quad and use mathematical functions to:

Draw a circle without adding a Sphere or Circle mesh.
Control its shape, size, and edges.
Combine multiple circles into a continuously changing liquid-like silhouette.

This effect is called Metaballs.

The underlying geometry stays the same, but the image changes as our mathematical rules change.

Metaballs

We can create visual complexity through computation, not just geometry.

Homework (due Nov 3.)

Shader Exploration

Complete the Unity Explore Shaders tutorial and experiment with a Shader Graph.

After completing the tutorial, create something new using Shader Graph.

You can use the tutorial or other examples as starting points, but try to develop your own visual effect rather than simply modifying a few properties. Focus on experimentation, creativity, and exploring what shaders can do.

Part 1. Read: Shaders and Materials

Creative Core: Shaders and materials

Review the course material and introductory sections, particularly:

Explore meshes and rendering
Explore shaders
Manage materials in a project

Focus on understanding the relationship between Meshes, Materials, Textures, and Shaders, and the difference between Vertex and Fragment Shaders.

You do not need to complete the entire Creative Core course.

Part 2. Practice: Create Your First Shader Graph

Get started with Shader Graph - Unity Learn

Follow the tutorial in Unity to create a custom shader using Shader Graph.

After completing the tutorial, experiment with your shader:

Modify at least two properties or node values.
Observe how these changes affect the visual result.
Try making the effect look different from the original tutorial.

Part 3. Application:

Using the Metaballs demo or any other shader examples as a starting point, create your own visual effect in Unity.

You are encouraged to experiment with, modify, and combine different techniques rather than simply reproduce an existing effect.

You do not need to build everything from scratch, but your final result should reflect your own creative decisions and meaningful modifications.

You will have an opportunity to demonstrate your work and share your creative process with the class.

Page updated

Report abuse