PAC-NeRF:

Physics Augmented Continuum Neural Radiance Fields

for Geometry-Agnostic System Identification

Xuan Li 1, Yi-Ling Qiao 2, Peter Yichen Chen 3,4, Krishna Murthy Jatavallabhula 4,

Ming Lin 2, Chenfanfu Jiang 1, Chuang Gan 5,6

1 UCLA, 2 UMD, 3 Columbia University, 4 MIT CSAIL, 5 UMass Amherst, 6 MIT-IBM Watson AI Lab

ICLR 2023 Notable-Top-25%

Overview of PAC-NeRF

PAC-NeRF is a novel approach to estimating both the unknown geometry and physical parameters of highly dynamic objects from multi-view videos. We design PAC-NeRF to only ever produce physically plausible states by enforcing the neural radiance field to follow the conservation laws of continuum mechanics.

For this, we design a hybrid Eulerian-Lagrangian representation of the neural radiance field, i.e., we use the Eulerian grid representation for NeRF density and color fields, while advecting the neural radiance fields via Lagrangian particles. This hybrid Eulerian-Lagrangian representation seamlessly blends efficient neural rendering with the material point method (MPM) for robust differentiable physics simulation.