Volume Rendering

Direct Volume Rendering by Ray Casting through Compositing

Dr. Yu Huang

Three-dimensional arrays of data are a convenient and widely-used representation for information. Medical imaging technologies such as magnetic resonance (MR) and computed tomography (CT) can produce 3D arrays of data containing detailed representations of internal organs, allowing doctors to make a diagnosis without invasive surgery. All of these measurement and simulation techniques can produce very large arrays of numbers that are difficult to understand due to the sheer quantity of data. Data visualization techniques specialized for large 3D arrays are necessary. One emerging technique is volume rendering, a method for producing an image from a 3D array of sampled scalar data.

The basic steps in any volume rendering algorithm consist of assigning a color and an opacity to each sample in a 3D input array, projecting the samples onto an image plane, and then blending the projected samples. The foundation of this visualization technique is a physically-based model for the propagation of light in a colored, semi-transparent material. The input to a volume rendering algorithm is a 3D array of scalars (as opposed to vectors which have a directional component). The array is called a volume and each element of the array is called a voxel.

Volume rendering is an approximate simulation of the propagation of light through a participating medium represented by the volume. The medium can be thought of as a block of colored, semi-transparent gel in which the color and opacity are functions of the scalar values in the input array. As light flows through the volume it interacts with the gel via several processes: light can be absorbed, scattered or emitted by the volume. Given a simplified optical model, a volume rendering algorithm produces an image by computing how much light reaches each point on an image plane.

However, with a different set of assumptions we arrive at the volume rendering equation. First, we only model single scattering, i.e. all of the photons reaching the image are assumed to have scattered only once after leaving a light source. Second, we ignore absorption between the light source and the scattering event (but not between the scattering event and the image plane). Third, we assume isotropic absorption. Fourth and finally, we choose a simple boundary condition: we assume that the only energy entering the volume comes from a finite set of point light sources.

A key advantage of volume rendering is that the volume data need not be thresholded, in contrast to surface rendering techniques. Surface rendering techniques for volume data operate by fitting polygons to an isosurface in the volume (using, for instance, the marching cubes algorithm, contour tracking, opaque cubes etc.), and then rendering the polygonal model with traditional polygon rendering techniques. The surface-fitting process requires making binary decisions, i.e. the volume must be thresholded to produce regions that are either inside or outside the isosurface. If the volume contains fuzzy or cloud-like objects then a polygonal surface will be a poor approximation. In contrast, volume rendering algorithms never explicitly detect surfaces so they naturally handle fuzzy data as well as sharply-defined surfaces.

What is the pipeline of the actions a user performs in a typical visualization session?

1) First the user acquires an array of input data. The acquisition process may include preparation steps such as resampling the volume to a regular grid, interpolating missing voxel values, and applying image processing operators to improve contrast.

2) Next the user classifies the volume. Classification is the process of assigning an opacity to each voxel. Proper choice of the classification function is difficult. A good classification function reveals structures in the volume or highlights some subset of the volume in an informative way.

3) After classifying the data the user chooses a shading function. The shading function specifies the illumination model and a rule for determining the color of each voxel. The rendering algorithm uses this information to compute.

4) Finally the user chooses viewing parameters: the viewpoint, the type of projection (parallel or perspective), clipping planes, and so on. With this information and the shaded, classified volume, the rendering algorithm produces an image.

At this point the user will almost certainly want to adjust the parameters to change or improve the image. Because of the subjective choices involved in classification and shading it is difficult to decide on a set of parameters a priori, so the user must rely on trial and error to produce a good visualization.

Existing volume rendering algorithms fall into four classes: ray casting algorithms, splatting algorithms, cell projection algorithms and multi-pass resampling algorithms. The two characteristics that distinguish each class are the order in which an algorithm traverses the volume and the method an algorithm uses to project voxels to the image. Each class of algorithms has performance advantages and disadvantages.

Ray casting algorithms produce an image by casting a ray through the volume for each image pixel and integrating the color and opacity along the ray. Ray casters are called image order algorithms since their outer loops iterate over the pixels in the image. Ray casters are also called backward projection algorithms since they calculate the mapping of voxels to image pixels by projecting the image pixels along viewing rays into the volume. Light rays flow forward from the volume to the image whereas viewing rays flow backward from the image into the volume.

In contrast to ray casting algorithms, splatting algorithms operate by iterating over the voxels. This class of algorithms computes the contribution of a voxel to the image by convolving the voxel with a filter that distributes the voxel’s value to a neighborhood of pixels, a process that is called splatting. Algorithms of this type are called object order algorithms since the outer loop iterates over voxels in the object being rendered. Splatters are also called forward projection algorithms since voxels are projected directly into the image, in the same direction as the light rays. In practice, since it is difficult to compute the filter weights in the splatting algorithm, approximations must be used. An alternative splatting algorithm uses 2D image warping techniques.

A third class of algorithms consists of the cell projection techniques. These methods are often used for volumes sampled on a non-regular grid. The first step is to decompose the volume into polyhedra whose vertices are the sample points (using a Delauney triangulation for instance). The algorithm then sorts the polyhedra in depth order and scan converts the polygonal faces into image space. Finally, the algorithm evaluates the volume rendering integral between the front and back faces of each polyhedron to compute a color and an opacity at each pixel, and these values are composited into the image. Cell projection algorithms are similar to splatting algorithms except that the former use polygon scan conversion to perform the projection.

The fourth class of volume rendering algorithms operates by resampling the entire volume to the image coordinate system so that the resampled voxels line up behind each other on the viewing axis in image space. The voxels can then be composited together along the viewing axis as in a ray caster, except that in the resampled volume the viewing rays are always axis-aligned. This class of algorithms uses multipass methods to resample the volume: the viewing transformation is factored into a sequence of simple shears and scales which are then applied to the volume in separate passes.

Figure: A Volume Rendering Tool with Ray Casting through Compositing for a X-ray Five View-based Explosive Detection System.

Appendix: Fast Volume Rendering by Shear-Warp Factorization

Several existing volume rendering algorithms operate by factoring the viewing transformation into a 3D shear parallel to the data slices, a projection to form an intermediate but distorted image, and a 2D warp to form an undistorted final image. In [1] they extend this class of algorithms in three ways. First, it describes a new object-order rendering algorithm based on the factorization that is significantly faster than published algorithms without loss of image quality. The algorithm achieves its speed by exploiting coherence in the volume data and the intermediate image. The shear-warp factorization can traverse both the volume and the intermediate image data structures in synchrony during rendering, using both types of coherence to reduce work. The second extension is a derivation of the factorization for perspective viewing transformations. Third, it introduces a data structure for encoding spatial coherence in unclassified volumes (i.e. scalar fields with no precomputed opacity). The algorithms employ run-length encoding, min-max pyramids, and multi-dimensional summed area tables. The method extends readily to support mixed volumes and geometry.