ST-PIO: A Method for Room Layout Estimation

Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Hao Zhao, Ming Lu, Anbang Yao, Yiwen Guo, Yurong Chen, Li Zhang

{zhao-h13@mails, lu-m13@mails, chinazhangli@mail}.tsinghua.edu.cn

{anbang.yao, yiwen.guo, yurong.chen}@intel.com

Above is the overview of conventional methods. Below is the overview of our method.

Top: Probabilistic node connectivity. Bottom: Semantic transfer. In stage three, pre-trained network refers to the one outlined by the dashed box in stage one. In stage four, pixel-wise edge labelling network refers to the one outlined by the dashed box in stage three.

(a) Network design for ST stage one. (b) Qualitative results for semantic segmentation on dataset LSUN. (c) Unsupervised structure visualization of the semantic feature space. (d) Transfer weights Visualization. Left-top: bg. Right-top: wf. Left-bottom: ww. Right-bottom: wc.

Feature Visualization and Comparison

Two core concepts about physics inspired optimization.

Left: qualitative results on LSUN validation set. The visualized feature map merges wf, ww, and wc by a pixel-wise max operation, yet they are used independently in PIO. Right: typical failure cases in which a wrong topology produces the lowest energy. See corresponding videos below.

Paper: https://drive.google.com/open?id=0B3MqJAwE5f3wSTB5eFJCOFdNZ1k

Supp: https://drive.google.com/open?id=0B3MqJAwE5f3wV3Z5TEJIYkthU2c

Videos: https://drive.google.com/open?id=0B3MqJAwE5f3waVc5c0RwcWI5Njg

Model: https://drive.google.com/open?id=0B3MqJAwE5f3wU25MZ0xjV0k3bzg

Hi-res figures: https://drive.google.com/open?id=0B3MqJAwE5f3wZWptQTZnQmJwX3c

Pre-computed ST features on LSUN test: https://drive.google.com/open?id=0B3MqJAwE5f3wMW56WERheThnMzQ