‘DiLiGenT’ Photometric Stereo Dataset

Introduction

Recent progress on photometric stereo extends the technique to deal with general materials and unknown illumination conditions. However, due to the lack of suitable benchmark data with ground truth shapes (normals), quantitative comparison and evaluation is difficult to achieve. We build the ‘DiLiGenT’ photometric stereo image dataset with calibrated Directional Lightings, objects of General reflectance, and ‘ground Truth’ shapes (normals) for orthographic projection and single-view setup.

In addition to the first dataset for such a purpose, we also provide a photometric stereo taxonomy emphasizing on non-Lambertian and uncalibrated methods. Based on our dataset, we quantitatively evaluate state-of-the-art photometric stereo methods for general non-Lambertian materials and unknown lightings to analyze their strengths and limitations.

Paper

Please cite the following paper if you use the data or the evaluation results reported in the paper:

Boxin Shi, Zhipeng Mo, Zhe Wu, Dinglong Duan, Sai-Kit Yeung, and Ping Tan, "A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo", In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Volume 41, Issue 2, Pages 271-284,2019. [PDF] [Supplementary Material]

(An earlier version in CVPR 16 can also be found here.)

Main dataset

The ‘DiLiGenT.zip’ package (850 MB) contains the following sub-folders:

‘pmsData’: The ‘DiLiGenT’ photometric stereo dataset with ten objects (as shown in the banner). For each object, we provide 16-bit integer PNG images with resolution of 612x512 from 96 different lighting directions. All evaluation results reported in the paper are calculated using the PNG images here. The mask image (‘mask.png’), lighting directions (‘light_directions.txt’, 3x96, with each row as a unit 3D vector), lighting intensities (‘light_intensities.txt’, 3x96, with each row represents the intensities in RGB channels; images are required to be normalized by dividing these intensity values per-channel before performing photometric stereo), and image file names (‘filenames.txt’) are provided within the subfolder of each object. The ‘ground truth’ normals are stored as Matlab matrix (‘Normal_gt.mat’), and a color-encoded normal map is provided as a reference (‘Normal_gt.png’).

‘sampleCodeBaseline’: An example of loading the images and running the baseline photometric stereo algorithm (simple linear least squares) using the ‘DiLiGenT’ dataset.

‘estNormalNonLambert’/‘estNormalUncalib’: Estimated per-pixel surface normal values stored as Matlab matrices by performing state-of-the-art non-Lambertian/uncalibrated photometric stereo methods on the ‘DiLiGenT’ dataset; please refer to Section 4 of the paper for details.

‘sampleCodeEvalNormal’: Sample codes that load the results in ‘estNormalNonLambert’/‘estNormalUncalib’ and calculate error statistics (based on per-pixel angular error and visualized using the box-and-whisker plot); the codes here reproduce Figures 2/4 of the paper.

Test dataset

The ‘DiLiGenT_Test.zip’ package (759 MB) has similar structure with the main dataset, but we do not include the 'ground truth normal' in 'pmsData' folder, and instead we provide the per-pixel angular error maps and statistics of angular errors. The test dataset contains the nine objects corresponding to #1--#9 in the main dataset under the same varying lighting condition, but they are taken from a different viewpoint.

‘pmsData’: Image and lighting format is the same as the main dataset, but the ‘Normal_gt.mat’ and ‘Normal_gt.png’ are not provided.

‘sampleCodeBaseline’, ‘estNormalNonLambert’, and ‘estNormalUncalib’ have similar contents as the main dataset.

'estNormalNonLambertErrMap' and 'estNormalUncalibErrMap': Pixel-wise angular difference maps (PNG images) of the estimated normal w.r.t. the 'ground truth' are provided; the numbers by the end of each file name are the mean angular errors. Statistics of errors (as box-and-whisker plot in EMF images, please refer to our paper for details) are also provided in the same folder.

Summary of benchmarking results

Click here to view a summary of benchmarking results on both main and test datasets.

Toolbox

The ‘shapeToImageAlignmentToolbox.zip’ package (8 MB) contains the code and instructions to perform the shape-to-image alignment and render the mesh to normal map. Given photometric stereo images and a scanned 3D mesh, the toolbox aligns them to the image coordinate system and renders the normal map as 'ground truth'.

Additional information

Reference BRDF -- ‘ACLSfitBRDFzip’ package (502 MB): We provide the BRDF fitting results using our image data, calibrated lightings, and the 'ground truth' normal. Sample code to reconstruct an image using the fitted BRDF is also provided. Please refer to the 'readme.txt' in the zip package for details.

Point light source positions -- 'PointLight.zip' package (9KB): We provide the point light source positions and camera intrinsic matrix in this zip package for evaluating photometric stereo with point light model and/or perspective camera model.

Benchmark evaluation

To get quantitative evaluation result on the test dataset for your own method, please send us the estimated normal maps (we encourage using the same format as the normal maps we provided for fast processing) via

photometricdataset =AT= gmail.com

and we will send back the results w.r.t. 'ground truth' (per-pixel difference maps). We are currently using a manual processing, and a fully automatic website with interative and real time feedback is still under construction.

The original high resolution (2448x2048), 32-bit floating-point EXR images and scanned 3D models of our objects (ply format) are available upon requests. And we always welcome feedback, suggestion or question regarding our dataset.