2017-07-12

First experimental results shown below with comparisons between Ground Truth (taken using complimentary codes), Normal Decoding, and Correlation Decoding with XOR codes. These results were taken in epipolar-only mode.

Here are the pictures of the objects involved in the experiments. It is difficult to get a direct picture of the object from the episcan camera itself due to bright reflections, and thus an alternate camera was used. The three objects involved in the experiments are a flat wall, two white cardboard boxes, and a melanine bowl.

Ground Truth Normal Decoding Correlation Decoding

The first row of bowl images were taken with the lights in the room on, while the second row of bowl images were taken with the lights in the room off. As seen above, in epipolar-only mode there are negligible differences.

2017-07-13

It was pointed out there may have been a few issues with the Episcan settings when taking the images above (images were worse than expected). I tweaked around the settings a bit along with Andrew Tan and after today I mainly fixed the issues with the Episcan (it turns out one of the projector wrap wasn't set properly after the Episcan's location was moved in the lab). More pictures will be taken on Monday, as I am unable to go to the lab on Friday. These are the improved results taken at the end of today:

Ground Truth Normal Decoding Correlation Decoding

2017-07-17

More pictures were taken of the improved results. As Parsa pointed out that the results can still be improved further, the Episcan was continued to be fine tuned today. In addition to this, I noticed that there may be a small error in the correlation decoding method as there are two distinct black lines running through every correlation image (these lines are especially clear in the flat wall images below). I've started working on fixing this error in the code but couldn't fix the error today.

Ground Truth Normal Decoding Correlation Decoding

*I've noticed I accidentally screen-shotted some of the computer within the images. I'll be careful not to do so in the future.

2017-07-18

Today was mainly spent preparing the code to work with the patterns Parsa optimized in the form of a .mat file. Several parts of the xorcode class functions applicable to all structured light patterns were relocated to the general structured light class. In addition to this, a new structured light class for new general .mat file patterns was created. Currently I'm working on how to integrate the .mat file and new class smoothly into the code. I also briefly looked through the correlation function today and is working on fixing the error (the error seems to be related to finding the best pixel match for pixels with a certain coded value).

Week - 2017-07-19, 2017-07-24 to 2017-07-28

As this week was mainly focused on writing code (and debugging it), the overall work and results of the week will be summarized and displayed below:

1) Pattern generation from a .mat file was implemented. The code itself uses the .csv file which can be obtained from the .mat file through a Python script.

2) Rewrote the structured light algorithm classes such that the decoding method can be directly passed in (instead of using several booleans etc.) as correlation decoding was added as a possible option. Also checked to ensure the functionality of old functions do not change due to this.

3) Finished implementing the new structured light class for displaying and decoding .mat files including pattern generation and correlation decoding. Correlation decoding currently has a few segmentation errors and will be fixed later.

4) Added new functions separate from the xorcode functions for displaying and capturing images of the patterns generated from the .mat files using the new structured light class in the main mySensor.cpp file.

As Parsa is currently running the algorithm to optimize the patterns for the projector's 1280px dimensions, I was temporarily given 6 patterns for 512px to use to test. The patterns generated from the .mat file are shown below:

Shown above is what the pattern look like when projected and captured by the camera.

Week - 2017-07-31 to 2017-08-04

As this week was also mainly focused on writing code (and debugging it), the overall work and results of the week will be summarized and displayed below:

1) Errors regarding the correlation decoding for the pattern generated from the .mat files were fixed. The function runs properly and generates a disparity map. However the results are not very good as the pattern file given is only 512px. Once Parsa finished optimizing the 1280px pattern, I can check the results again to ensure the correlation is working properly.

2) Some adjusting of the Episcan was done to generate better images. Pictures were taken for Episcan and Conventional modes. There is still the error of the line running through depth maps created by the correlation decoding. However, as I can also see this line in some results from standard xorcode decoding, I am unsure of whether this line is related to a problem in the code or perhaps a problem in the image capturing.

3) A function was written to generate a mismatch image with banding, as well as to calculate the RMSE between the inputted obtained disparity map and ground truth. Results are shown below.

Ground Truth Normal Decoding Correlation Decoding

The first row of images are taken in conventional mode, while the second row of images are taken in Episcan mode. The captured object is the same melanine bowl as used in earlier taken images. The adjusting doesn't seem to have particularly improved the image quality to a great degree. What is very interesting to note is that the odd line in the correlation decoding images can be seen very clearly in depth map for the normal decoding in conventional mode (this also made me realize the same line is present in normal decoding in Episcan mode, although very faint). As a result, I'm not sure if this line is related to a problem in the code or perhaps a problem in the image capturing.

Band 1 Band 3 Band 5

Shown above is the Mismatch image results with different bands between the two depths maps shown on the bottom row. On the left is the Ground Truth and on the right is the map generated by correlation decoding. There is a minor issue with the color (according to the generated .txt file of the image matrix, all values are either 0 or 255 and there shouldn't be any color), but this may be an error with how the Mat object is saved as a .png file in the code.

Shown above is a second set of mismatch images between another set of depth map images. These images do not have the accidental screen show icons displayed on the bottom. The RMSE calculated for these two images is 10.0832. (The RMSE function was not yet implemented when the first set of mismatch images were taken).