Post Pipeline
Post Pipeline
Embedded Files
The output of Predict Engine is a RAW image, similar to the one you could get by taking a photo with an actual camera. In order to display this image on screen, a post-process must be carried out, mostly in order to adjust the image dynamic range. This post-process is defined in the Post Pipeline section of the UVR Camera Settings component.
The post pipeline consists of the following main steps :
Converting the sensor raw image to a displayable RGB image defining a color system,
For photographic sensors, defining the reference white in the image,
Denoising the image,
Adjusting the Dynamic Range of the image using a tone mapper or false colors.
The post pipeline can be edited at runtime via the Interactive Settings interface.
Color System
Color System
The raw HDR output image from Predict Engine must be converted to a displayable LDR RGB image. The output image can be either :
the raw image converted to LDR RGB using a given gamut,
the raw image with a gamma correction only, or
the raw image as a representation of the measured physical quantity in false colors.
Gamma Correction
Gamma Correction
You can find a good explanation of what the gamma correction is needed for on this page. Here is an extract :
"The need for gamma arises for two main reasons: The first is that screens have a non-linear response to intensity. The other is that the human eye can tell the difference between darker shades better than lighter shades. This means that when images are compressed to save space, we want to have greater accuracy for dark intensities at the expense of lighter intensities. Both of these problems are resolved using gamma correction, which is to say the intensity of every pixel in an image is put through a power function. Specifically, gamma is the name given to the power applied to the image.
Most images are stored with a gamma of 0.45 (1/2.2) applied to them [...]. Images stored like this are in “gamma space” [...]. This is the default behavior of nearly every digital camera, as well as image editing applications and so on. In fact, nearly every image you see on your computer has that gamma applied to it.
You may be wondering why images display correctly, and don’t look too bright. This is where the non-linear response of displays comes in. CRT screens, simply by how they work, apply a gamma of around 2.2, and modern LCD screens are designed to mimic that behavior. A gamma of 2.2, the reciprocal of 0.45, when applied to the brightened images will darken them, leaving the original image."
In order to display on screen an image corrected for your specific screen, Predict Engine needs the exact value of this gamma correction : this is the gamma defined in the color system.
Gamma = 0.45
Gamma = 1
Gamma = 2.2
Unity supports two kinds of color space : "Linear" and "Gamma", defined in the Project Settings/Player, section Other Settings :
If the color space is set to "Linear", Unity already takes into account the screen gamma correction and the gamma correction in Predict Engine should be set to 1,
If the color space is set to "Gamma", you will get the behaviour described above.
The gamma value defined in the color system should match your display. Follow the next steps to find the right gamma for your display :
Download the image on the right on your computer (unless you have a truecolor video system, do not attempt to use the image while viewing it online, since your browser will be changing the colors to compensate for other colors on the screen),
Open the image with a viewer that displays the correct intensities (ensure that the image is not enlarged or reduced in any way by the viewing program, as this will also ruin its effectiveness),
Darken the room and set the monitor brightness and contrast to maximum,
While viewing a black screen, lower the brightness gradually until the "background" (ie the scanlines that you see across what should be a black screen) is no longer noticeable (when it just fades from view). This may be difficult on monitors that normally cover the whole visible portion of the display, unless you change the viewable area settings until you can see the actual border where the monitor isn't displaying anything,
While viewing the reference image, lower the contrast until the alternating white and black bars on the left edges of each column are equal in width. This is trying to get a 50% gray by using half white and half black. If this is not possible, choose a contrast setting which is about in the middle of the available range,
While viewing the image from a distance, or with squinted eyes, one of the numbered "swatches" will best match the gray value approximated by the white and black bars. The number in this "swatch" is your display's actual gamma value (usually 2.2).
Display gamma reference image
Additional color system settings are available if you enable the Expert Mode.
White Balance
White Balance
When using Photographic sensors, just like with a regular camera, white balance may be required. You can define the color that will be used as the reference white in the balance.
In the renderer view interface, an additional button next to the White Reference field enables you to pick a reference value directly in the Predict Engine render. Using this picker, the selected color will be the RGB value in the Predict Engine render before every post process (white balance included) : this option is especially useful if the current white reference is not white.
White balance can be enabled/disabled and the white reference can be edited at runtime.
Denoiser
Denoiser
Path tracing can induce noise in the generated image. The longer an image takes to compute, the less noise there is. To reduce the noise while the image is rendering, a denoiser can be used.
The denoiser can be enabled/disabled at runtime.
Additional denoiser settings are available if you enable the Expert Mode.
Tone Mapper
Tone Mapper
Images produced by Predict Engine are defined within a dynamic range that cannot be displayed on the screen : values on screen must be contained within [0;255] whereas values in Predict Engine can go up to thousands or be contained in a much smaller interval. In order to adapt this dynamic range, we use a Tone Mapper.
The tone mapper used in Predict Engine is linear : a simple multiplicative factor is applied to the image. This factor is called the Exposure Value.
The Exposure Value can be chosen manually or it can be computed automatically (when the Auto Exposure field is enabled). When the auto exposure mode is enabled, the exposure value is computed so that the average of a zone at the center of the image is around 128 (half the maximum value). The EV +/- field enables you to make the image darker or brighter by changing this expected value.
When the auto exposure is enabled, 3 additional buttons appear next to the Auto Exposure field :
The pen button on the left defines whether the zone used to compute the average is circled on the image or not,
The pipette button on the right enables you to pick a zone on the image that will be use to compute the average (this option is only available when using the Engine view),
The square button in the middle reset the zone used to compute the average to the center of the image.
Scene previewed using different auto exposure zones, the zone used to compute the exposure value is placed on the ceiling light (left) or on the red sphere (right)
The tone mapping is not applied to false color images because they are already a RGB visualization of an HDR value.
The tone mapper settings can be edited at run time.
False Colors
False Colors
When using a False Color Color System, the raw output image from Predict Engine is visualized as a representation of the measured physical quantity in false colors.
Photometric sensors output an image with XYZ values. The Y channels corresponds to the perceived luminance in cd/m². The channel displayed on screen can be selected in the Color System section.
Radiometric sensors output a value in W/m²/sr. You can define which channel is displayed in the Color System section.
If the Auto Range is enabled, the color map will represent all the values in the image. If it is disabled, a "Range" field enables you to define the range of values (in %) that should be represented. This enables you to ignore the lowest/highest values in the image that may correspond to fireflies/noise or saturated zones with direct illumination.
You can choose to display the raw value or the logarithm of the value.
You can choose to show the scale on screen or not.
Polarimetric sensors are necessarily in false colors as the raw output image from Predict Engine cannot be interpreted as an RGB visualisation of the scene. Six visualization modes are available for the polarization in addition to the four Stokes components :
The Degree mode displays the degree of polarization in the range [0;1] using a red scale colormap (see Figure B bellow),
The Orientation mode displays the orientation of the polarized light, in the range [0;180] degrees, using a Rainbow colormap (see Figure D bellow),
The Ellipticity mode displays the ellipticity of the circularly polarized light, in the range [-45;45] degrees : an ellipticity of +45° (right) is represented by the red color, an ellipticity of -45° (left) is represented by the blue color, and an ellipticity of 0° is represented by the black color (see Figure F bellow),
The Type mode displays the type of the polarization : linear polarization is represented by the cyan color and circular polarization is represented by the yellow color (see Figure C bellow),
The Chirality mode displays the chirality of the circularly polarized light (S3 component of the Stokes vector) : a right chirality is represented by the blue color and a left chirality is represented by the yellow color (see Figure E bellow),
The Plane mode displays the orientation of the linearly polarized light (circularly or elliptically polarized light will be ignored), in [0;180] degrees, using a red/green colormap for the S1 Stokes component and a blue/yellow colormap for the S2 Stokes component (see Figure G bellow),
The S0 mode display the raw S0 component of the Stokes vector,
The S1 Reduced, S2 Reduced, and S3 Reduced modes display the reduced components of the Stokes vector : the raw component divided by S0.
More details on the visualisation modes of the polarization :
Wilkie, A. and Weidlich, A. (2010). A standardised polarisation visualisation for images. In Proceedings of the 26th Spring Conference on Computer Graphics, pages 43–50. ACM.
The false color settings can be edited at run time.
Example scene defined with three spheres (one red diffuse, one metallic and one glass), a mirror in the background, and two polarizers on the front (linear polarizer on the left, circular polarizer on the right), lit by a D65 ambient light.
Figure A : scene rendered with a photographic sensor
Figure B : scene rendered in the Degree mode.
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear/circular polarizer is purely linearly/circularly polarized.
The diffuse material is not inducing polarization.
Figure C : scene rendered in the Type mode
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear/circular polarizer is purely linearly/circularly polarized.
The diffuse material is not inducing polarization.
Figure D : scene rendered in the Orientation mode.
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear polarizer is purely linearly polarized in a single direction.
Figure E : scene rendered in the Chirality mode.
The light going through the circular polarizer is purely circularly polarized with one chirality.
Figure F : scene rendered in the Ellipticity mode.
The light going through the circular polarizer is purely circularly polarized with one chirality.
Figure G : scene rendered in the Plane mode
The reflection/transmission on the mirror and the dielectric spheres induce linear polarization.
The light going through the linear polarizer is purely linearly polarized in a single direction.
Figure H : scene rendered in the S0 mode.
Figure I : scene rendered in the S1 Reduced mode.
Figure J : scene rendered in the S2 Reduced mode.
Figure K : scene rendered in the S3 Reduced mode
Page updated
Report abuse