This project focuses on Neural Radiance Fields (NeRF) and volumetric rendering techniques, which enable the generation of realistic 3D scenes from 2D images by modeling how light interacts within a volume. NeRF represents scenes as continuous volumetric fields that map 3D coordinates and viewing directions to color and density, while volumetric rendering simulates light absorption and emission along rays passing through these volumes.  

The main objectives are to implement and optimize differentiable rendering pipelines that reconstruct detailed geometry and appearance, enabling photorealistic novel view synthesis and accurate depth estimation from sparse image data.

Differentiable Volume Rendering

The core of the project is the implementation of an emission-absorption volume renderer that is differentiable end-to-end, allowing learning and optimization from images alone. The rendering process simulates how light accumulates color and looses intensity through a volume along camera rays.

• Ray Generation: Mapping 2D pixels to 3D rays emanating from camera positions.

• Stratified Sampling: Collecting sample points along each ray between near and far planes, to improve coverage and reduce bias.

• Volume Evaluation: Computing density and emitted radiance  at every sample point by querying an implicit function representing the scene.

• Aggregation: Using the emission-absorption model, colors and opacities are integrated along the ray, weighted by transmittance and volume density, according to

where Ti denotes transmittance up to sample i, Δti is the distance between samples, and the term captures how much light is contributed by each segment. 

This formulation enables gradients to flow through rendering, supporting inverse rendering and optimization. 

The basic implicit volume, represented as an SDF volume  box), is implemented in the SDFVolume class found in implicit.py.

Optimizing a Basic Implicit Volume

Using the differentiable renderer, the project next tackled optimizing simple 3D shapes. A box’s position and dimensions were refined to closely match ground-truth camera images using gradient descent. Instead of rendering every pixel each iteration, a random subset (mini-batch) of rays was sampled to manage computational resources efficiently. The loss was defined as the mean squared error between predicted and true pixel colors, driving the optimization. The resulting renderings show the box geometry converging steadily towards the reference, confirmed by smooth spiral views.

Optimizing a Neural Radiance Field (NeRF)

Constructed a Multi-Layer Perceptron (MLP) network that models a Neural Radiance Field and maps 3D spatial coordinates to volume density and color, effectively learning the scene’s volumetric representation from images alone. Positional encoding was applied to inputs to capture fine details and high-frequency variation. The NeRF model is implemented in the NeuralRadianceField class within the file implicit.py.

Training on the LEGO bulldozer dataset, the NeRF achieved accurate novel view synthesis- rendering smooth perspective changes and intricate geometry without explicit 3D supervision. The network output non-negative densities using ReLU and colors using Sigmoid activations, ensuring physically plausible values. 

To better capture real-world lighting, view-dependent effects were added by conditioning the MLP’s output on viewing direction. This enabled the model to represent phenomena such as specular reflections and appearance changes with camera angle. The model was then trained trained and evaluated on the Materials dataset, and side-by-side comparisons showed significant improvements in visual realism with view dependence.

Sphere Tracing

Sphere tracing was implemented to efficiently render surfaces defined implicitly by SDFs, avoiding exhaustive volumetric sampling. The algorithm iteratively moves rays forward by the minimum distance to a surface, as given by the SDF value. This approach rapidly approximates ray-surface intersections with high accuracy. The SDF model is implemented in the NeuralSurface class, also located in implicit.py

Applied to a torus, sphere tracing produced smooth, clean renderings without artifacts. This method serves as a powerful foundation for rendering implicit geometries interactively.

Optimizing a Neural SDF

Trained an MLP to represent a neural SDF, modeling the signed distance to surfaces from point cloud data. An eikonal loss was applied to regularize gradients of the SDF to have unit norm, ensuring geometric consistency and smoothness. 

Optimization produced detailed surface reconstructions from sparse data, demonstrating the network’s ability to infer smooth implicit geometry from limited input. 

VolSDF

Building on neural SDFs, the VolSDF framework was implemented to predict color at each point and convert SDF values to volume densities for volume rendering. The critical density mapping function, parameterized by α and β, shapes the transition sharpness between free space and surfaces. Larger β produces sharper surfaces but may make optimization trickier; smaller β encourages smoothness at the cost of crispness.

Trained on the LEGO scene, VolSDF generated realistic color and geometry renderings with clear surface boundaries, integrating surface and volume rendering techniques into a unified framework.

Key Outcomes

• Built a ray‑sampling pipeline that renders both color and depth by simulating how light travels through a volume.

• Implemented a NeRF to learn 3D shape and appearance from images, and extended it by integrating view-dependance using camera‑angle conditions to capture realistic specular and reflective effects.

• Developed an SDF‑based ray tracing method that efficiently and accurately renders smooth surfaces.

• Trained a neural SDF with eikonal regularization, then added color and density mapping for photorealistic surface rendering and robust results on complex and sparse-data scenes.

Conclusion

This project highlights how modern neural rendering methods can learn and reconstruct detailed 3D shapes and scenes directly from 2D images. It demonstrates state‑of‑the‑art capabilities in generating realistic 3D representations from limited data using deep learning with efficient rendering techniques such as differentiable volume rendering, neural radiance fields, sphere tracing, and VolSDF. This project lays the groundwork for future advances in graphics and vision.