We investigated disparity estimation from a light field using a convolutional neural network (CNN). Most of the methods used for this task implemented a supervised learning framework, where the predicted disparity map was compared directly to the corresponding ground-truth disparity map in the training stage. However, light field data accompanied with ground-truth disparity maps were insufficient and rarely available for real-world scenes. The lack of training data resulted in limited generality of the methods trained with them. To tackle this problem, we took a simple plug-and-play approach to remake a supervised method into an unsupervised (self-supervised) one. We replaced the loss function of the original method with one that does not depend on the ground-truth disparity maps. More specifically, our loss function is designed to indirectly evaluate the accuracy of the disparity map by using weighted warping errors among the input light field views. Our loss function was also designed to be aware of the edge alignment between the image and the disparity map. As a result of this unsupervised learning framework, our method can use more abundant training datasets (even those without ground-truth disparity maps) than the original supervised method. Our method was evaluated on computer-generated scenes (4D Light Field Benchmark) and real-world scenes captured by Lytro Illum cameras. Our method achieved the state-of-the-art performance as an unsupervised method on the benchmark. We also demonstrated that our method can estimate disparity maps more accurately than the original supervised method for various real-world scenes.
Taisei Iwatsuki, Keita Takahashi, Toshiaki Fujii: "Unsupervised Disparity Estimation from Light Field Using Plug-and-Play Weighted Warping Loss", EURASIP Signal Processing: Image Communication, 2022. [ publisher's website (open access) ]
Software (written by Python and Tensorflow): please refer to "readme.txt" for the term of use and usage. [Download]
Thanks to the excellent learning capability of deep convolutional neural networks (CNN), monocular depth estimation using CNNs has achieved great success in recent years. However, depth estimation from a monocular image alone is essentially an ill-posed problem, and thus, it seems that this approach would have inherent vulnerabilities. To reveal this limitation, we propose a method of adversarial patch attack on monocular depth estimation. More specifically, we generate artificial patterns (adversarial patches) that can fool the target methods into estimating an incorrect depth for the regions where the patterns are placed. Our method can be implemented in the real world by physically placing the printed patterns in real scenes. We also analyze the behavior of monocular depth estimation under attacks by visualizing the activation levels of the intermediate layers and the regions potentially affected by the adversarial attack.
Koichiro Yamanaka, Ryutaroh Matsumoto, Keita Takahashi, Toshiaki Fujii: "Adversarial Patch Attacks on Monocular Depth Estimation Networks", IEEE Access, Vol. 8, pp. 179094--179104, DOI: 10.1109/ACCESS.2020.3027372 (2020.9.28). [ Publisher's website ]
Our source codes written in Python + PyTorch [ Get our software (79.7MB)]
This package includes the source codes for training, testing, and visualization of our method. To find the usage and terms of use, please refer to "readme.txt".