3D Shape Analysis and Retrieval - Recent Advances and Future Trends

In the International Conference on Pattern Recognition (ICPR) 2012

By:

  • Hamid Laga, School of Mathematics and Statistics, University of South Australia, Australia.

  • Ryutarou Ohbuchi, Computer Science and Engineering Department, University of Yamanashi, Japan.

Abstract: In recent years, the acquisition and modelling of 3D data have gained a significant boost due to the availability of commodity devices. Digital 3D shape models are becoming a key component in many industrial, entertainment and scientific sectors. Consequently, large collections of 3D data are nowadays available both in the public (e.g., on the Internet) as well as in private domains. Analysing, classifying, and querying such 3D data collections are becoming topics of increasing interest in computer vision, pattern recognition, computer graphics, and digital geometry processing communities. 3D shape analysis poses new challenges that are not existent in image and video analysis. The purpose of this tutorial is to introduce the foundation of this topic to the pattern recognition community and overview the state-of-the-art techniques. The tutorial will start by introducing basic concepts such as 3D shape representations and shape descriptors while outlining the major requirements and challenges. Then the tutorial looks at the fundamental problem of comparing shapes, where one seeks to design similarity measures that capture shape properties (ranging from geometry to semantics), and which are robust to different variabilities (such as non-rigid deformations). We will also discuss roles that machine learning plays in 3D shape analysis and retrieval. Then, we will review recent works on query specification for 3D retrieval. We conclude the tutorial with an overview of some (classical and non-classical) applications where 3D shape analysis plays a central role.

Course outline

    1. Introduction (PDF)

    2. Rigid 3D Shape Analysis (PDF)

    3. Non-rigid 3D Shape Analysis (PDF)

    4. Manifold-based Analysis of Deformable Surfaces (PDF)

    5. Learning (PDF)

    6. Querying 3D Shape Databases (PDF)

    7. Applications (PDF)

    8. Summary (PDF)

References (PDF)

I. Descriptor-based 3D shape analysis

# Global descriptors

  • R. Osada, T. Funkhouser, Bernard Chazelle, and David Dobkin. Shape Distributions. ACM TOG, 21(4): 807-832, 2001

  • Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen and Ming Ouhyoung. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum (EUROGRAPHICS'03), 22(3):223-232, Sept. 2003.

  • Hamid Laga, Hiroki Takahashi and Masayuki Nakajima. Spherical Wavelet Descriptors for Content-based 3D Model Retrieval. IEEE International Conference on Shape Modeling and Applications (SMI) 2006.

  • Hamid Laga, Masayuki Nakajima, Kunihiro Chihara. Discriminative Spherical Wavelet Features for Content-based 3D Model Retrieval. International Journal on Shape Modeling, 13(1):51-72, 2007.

  • Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. Rotation Invariant Spherical Harmonic Representation of 3D. Shape Descriptors. Symposium on Geometry Processing, pp.156-164, 2003.

# Local descriptors

  • Andrea Frome, Daniel Huber, Ravi Kolluri, Thomas Bulow, and Jitendra Malik. Recognizing Objects in Range Data Using Regional Point Descriptors. Proc. European Conference on Computer Vision (ECCV), May 2004.

  • N. Gelfand, N. Mitra, L. Guibas and H. Pottmann. Robust Global Registration. Proc. 2005 Symposium on Geometry Processing, pp. 197-206, 2005.

  • Johnson, A. E., Hebert, M. Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21, 433–449, 1999.

  • Ryutarou Ohbuchi, Kunio Osada, Takahiko Furuya, Tomohisa Banno. Salient local visual features for shape-based 3D model retrieval. Proc. IEEE Shape Modeling International, pp. 93-10, 2008.

  • Jan Knopp, Mukta Prasad, Geert Willems, Radu Timofte, and Luc Van Gool. Hough Transform and 3D SURF for robust three dimensional classification. Proc. European Conference in Computer Vision (ECCV), pp.589-602, 2010.

    • Tal Darom and Yosi Keller. Scale Invariant Features for 3D Mesh Models. IEEE Trans. Image Processing, 21(5), pp.2758-2769, 2012.

II. Shape similarity

# Bag of words

  • Takahiko Furuya, Ryutarou Ohbuchi. Dense sampling and fast encoding for 3D model retrieval using bag-of-visual features. Proc. ACM CIVR 2009, 2009.

  • M. Bronstein, M. M. Bronstein, M. Ovsjanikov, L. J. Guibas. Shape Google: geometric words and expressions for invariant shape retrieval. ACM Trans. Graphics (TOG), Vol. 30/1, pp. 1-20, January 2011.

# Isometry invariant shape analysis

  • Ran Gal, Ariel Shamir, Daniel Cohen-Or. Pose oblivious Shape Signature. IEEE Transactions on Visualization and Computer Graphics 2007.

  • Varun Jain, Hao Zhang, and Oliver van Kaick. Non-Rigid Spectral Correspondence of Triangle Meshes. Int. Journal on Shape Modeling (2007).

  • Varun Jain and Hao Zhang. A Spectral Approach to Shape-Based Retrieval of Articulated 3D Models. Computer-Aided Design, Vol. 39, Issue 5, pp. 398-407, 2007.

# Diffusion Geometry

  • Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. A Concise and Provably Informative Multi-scale Signature Based on Heat Diffusion. Proc. Eurographics Symposium on Geometry Processing (SGP) 2009.

  • M. Reuter, F.-E. Wolter and N. Peinecke. Laplace-Beltrami spectra as "Shape-DNA" of surfaces and solids. Computer-Aided Design 38 (4), pp.342-366, 2006.

  • Iasonas Kokkinos, Michael Bronstein, Roee Litman, Alexander Bronstein. Intrinsic Shape Context Descriptors for Deformable Shapes. CVPR 2012.

# Elastic shape analysis

  • S. Kurtek, E. Klassen, J. Gore, Z. Ding, and A. Srivastava. Elastic Geodesic Paths in Shape Spaces of Parameterized Surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence (2011).

III. Some semantics

IV. Supervised / unsupervised Learning for 3D shape analysis and retrieval

  • George Leifman, Ron Meir and Ayellet Tal. Semantic-oriented 3D shape retrieval using relevance feedback. The Visual Computer, 21 (8-10) (2005), pp. 865-875

  • Ryutarou Ohbuchi, Jun Kobayashi. Unsupervised learning from a corpus for shape-based 3D model retrieval. Multimedia Information Retrieval 2006: 163-172

  • Philip Shilane and Thomas Funkhouser. Distinctive Regions of 3D Surfaces. ACM Transactions on Graphics, 26(2), June 2007.

  • Hamid Laga. Automatic Selection of Best Views of 3D Shapes. The Visual Computer 2011.

V. Querying 3D shape databases

  • Mathias Eitz, Ronald Richter, Tamy Boubekeur, Kristian Hildebrand and Marc Alexa. Sketch-based Shape Retrieval. Transactions on Graphics, Proc. SIGGRAPH 2012.

  • Mathias Eitz, James Hays, Marc Alexa. How Do Humans Sketch Objects? Transactions on Graphics, Proc. SIGGRAPH 2012.

  • M. Fisher, P. Hanrahan. Context-based Search of 3D Models. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2010).

  • Corey Goldfeder, Peter K. Allen. Autotagging to Improve Text Search for 3D Models. Joint Conference on Digital Libraries 2008.

  • M. Fisher, M. Savva, P. Hanrahan. Characterizing Structural Relationships in Scenes Using Graph Kernels. SIGGRAPH 2011.

  • T.F. Ansary, J-P. Vandeborre, Mohamed Daoudi. 3D-Model Search Engine from Photos. Proc. ACM CIVR 2007, pp. 89-92, 2007.

VII. Applications:

  • T. Funkhouser, M. Kazhdan, P. Shilane, P. Min, A. Tak. S. Rusinkiewicz, and D. Dobkin. Modeling by example. ACM TOG (23)3:652-663. (Proc. SIGGRAPH 2004), 2004.

  • Qi-Xing Huang. Simon Flöry, Nathsha Gelfand, Michael Hofer and Helmut Pottmann. Reassembling Fractured Objects by Geometric Matching. ACM TOG 25(3):569-578,(Proc. SIGGRAPH 2006), 2006.

  • Vladimir G. Kim, Wilmot Li, Niloy Mitra, Stephen DiVerdi, Thomas Funkhouser. Exploring Collections of 3D Models using Fuzzy Correspondences. ACM TOG 31(4), (Proc. SIGGRAPH 2012), 2012.

  • Slvie Philipp-Foliquet, Michael Jordan, Laurent Najman, Jean Cousty. Artwork 3D model database indexing and classification. Pattern Recognition, 44(3), March 2011, pp.588-597.

  • F. Shulze, M. Trapp, K. Buhler, T. Liu, B. Dickson. Similarity Based Object Retrieval of Composite Neuronal Structures. Proc. Eurographics 3DOR 2012.

VIII. Datasets and benchmarks

  • The Princeton Shape Benchmark: http://shape.cs.princeton.edu/benchmark/

  • Shape Retrieval Evaluation Context (SHREC): http://www.aimatshape.net/event/SHREC

  • The TOSCA datasets: http://tosca.cs.technion.ac.il/book/resources_data.html

  • Shape Retrieval Contest: http://www.aimatshape.net/event/SHREC

IX. Other (related / similar) tutorials