Nowadays, many cities around the world maintain semantic 3D city models. These models are characterized by rich semantics, high global accuracy but, too, mostly coarse geometrical representation due to the used geodata acquisition technique. The recent influx of point clouds acquired by mobile mapping units enables new possibilities for 3D modeling such as detailed building façades reconstruction.

This, however, is still an open challenge due to MLS point clouds drawbacks such as massive datasets, unordered dataset structure, or proneness to occlusions. These features hamper 3D geometry reconstruction, which requires high computational power and topological constraints. Moreover, to enable semantic 3D geometry reconstruction, a point cloud requires semantic segmentation, too.

On the other hand, there exist databases that contain already reconstructed 3D models of facade details (e.g., windows or doors): crowdsourced CAD models [Trimble Inc, 2021] that offer an enormous library of façades variations. The library can yield not only descriptors but the most probable representation of a façade, too.

The Bag-of-words (BoW) approach [Csurka et al., 2004] can be utilized to create an efficient 3D reconstruction method based on CAD models and point clouds. As the BoW approach has been successfully applied in 3D shape retrieval [Lian et al., 2010], among others. Such enriched buildings might be utilized in autonomous driving simulations or urban planning.

References:

Csurka, G, Dance, C, Fan, L, Willamowski, J and Bray, C, (2004) Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV Vol. 1, No. 1-22, pp. 1-2.

Lian, Z, Godil, A and Sun, X, (2010) Visual similarity based 3D shape retrieval using bag-of-features. In IEEE 2010 Shape modeling international conference, 21-23 June 2010, Aix-en-Provence, France pp. 25-36.

Memon, SA, Akthar, F, Mahmood T, Azeem M and Shaukat, Z, (2019) 3D Shape Retrieval using Bag of Word Approaches, 2nd IEEE International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), 30-31 January 2019, Sukkur, Pakistan, 2019, pp. 1-7, doi: 10.1109/ICOMET.2019.8673397.

Trimble Inc (2021) SketchUp 3D Warehouse. https://3dwarehouse.sketchup.com/ (4 February 2021)

For the training and validation of machine learning models, one needs to have a reliable source of ground-truth information. Currently, most of the ground-truth annotations for Mobile Laser Scanning (MLS) point clouds are acquired manually, which is both a laborious and tedious task that limits solutions' scalability. Moreover, even manually annotated datasets mostly omit classes depicting a building’s details, such as windows or doors, which excludes them from applications regarding 3D models reconstruction.

Nowadays, many cities maintain highly reliable 3D models characterized by high global accuracy and rich semantics at different levels of detail (LoD). Therefore, the existing semantic about a scene can be transferred from semantic 3D city models to MLS point clouds to facilitate the annotation process.

References:

Biljecki, F, Stoter, J, Ledoux, H, Zlatanova, S and Çöltekin, A, (2015) Applications of 3D city models: State of the art review. ISPRS International Journal of Geo-Information, 4(4), pp. 2842-2889

Griffiths D, Boehm J, (2019) A review on deep learning techniques for 3D sensed data classification. Remote Sensing, 11: 1499–1528

Schwab B, Haas-Goschenhofer S and Wysocki O, (2020) LoD3 Road Space Models. https://github.com/savein/lod3-road-space-models (31 January 2021)

Zhu J, Gehrung J, Huang R, Borgmann B, Sun Z, Hoegner L, Hebel M, Xu Y, Stilla U (2020) TUM-MLS-2016: An annotated mobile LiDAR dataset of the TUM city campus for semantic point-cloud interpretation in urban areas. Remote Sensing, 12(11): 1875

Point clouds are deemed as one of the best dataset types for urban mapping purposes. Yet, few researchers have addressed the use of point clouds for façade-level segmentation. Efficient façade segmentation is becoming pivotal in various applications ranging from simulating autonomous driving functions to preserving cultural heritage. [Wysocki et al., 2022]

Combining geometric features and deep learning networks has proved successful on cultural heritage sites [Grilli & Remondino, 2020]. However, their impact on a common urban scenarios is yet to be analysed.

Your task will encompass: testing performance of selected state-of the-art networks on typical urban façades; improving networks' performance by adding selected geometric features.

References:

Griffiths D, Boehm J, (2019) A review on deep learning techniques for 3D sensed data classification. Remote Sensing, 11: 1499–1528

Grilli, E., Farella, E. M., Torresani, A., & Remondino, F. (2019). Geometric feature analysis for the classification of cultural heritage point clouds. In 27th CIPA International Symposium “Documenting the past for a better future” (Vol. 42, pp. 541-548).

Grilli E, Remondino F. (2019) Classification of 3D Digital Heritage. Remote Sensing. 11(7):847. https://doi.org/10.3390/rs11070847

Grilli E, Remondino F. (2020) Machine Learning Generalisation across Different 3D Architectural Heritage. ISPRS International Journal of Geo-Information. 9(6):379. https://doi.org/10.3390/ijgi9060379

Wysocki, O., Hoegner, L., and Stilla, U.: . TUM-FACADE: Reviewing and enriching point cloud benchmarks for facade segmentation., Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLVI-2/W1-2022, 529–536, https://doi.org/10.5194/isprs-archives-XLVI-2-W1-2022-529-2022, 2022.

Zhu J, Gehrung J, Huang R, Borgmann B, Sun Z, Hoegner L, Hebel M, Xu Y, Stilla U (2020) TUM-MLS-2016: An annotated mobile LiDAR dataset of the TUM city campus for semantic point-cloud interpretation in urban areas. Remote Sensing, 12(11): 1875

Nowadays, many cities around the world create semantic 3D city models using aerial observations. However, these models have mostly coarse geometrical representation. As more and more mobile laser scanning (MLS) units capture road space environments, datasets barely available before are emerging. This enables the reconstruction of 3D models details, such as façades, which can improve autonomous driving simulations, solar potential analysis, and urban planning.

Yet, the Mobile Laser Scanning (MLS) technique has several challenges to overcome when it comes to building reconstruction. One of the pivotal ones is the incomplete building coverage owing to the acquisition geometry of such street-level mobile scanners.

Model-driven approaches have potential to overcome such data flaws. Yet, such approaches require appropriate models and suitably segmented point cloud to the task at hand.

Your tasks will be to: create library of underpass-related primitives; develop a clustering process for underpasses given some semantic priors; fitting the primitives into clusters.

References:

Balado Frías, J, Gonzalez Rodriguez, ME, Verbree, E, Díaz Vilariño, L and Lorenzo Cimadevila, HR, (2020) Automatic detection and characterization of ground occlusions in urban point clouds from mobile laser scanning data

Biljecki, F, Stoter, J, Ledoux, H, Zlatanova, S and Çöltekin, A, (2015) Applications of 3D city models: State of the art review. ISPRS International Journal of Geo-Information, 4(4), pp. 2842-2889

Goebbels S, Pohle-Fr ̈ohlich R, Pricken P (2019) Iterative Closest Point algorithm for accurate registration ofcoarsely registered point clouds with CityGML models. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2/W5: 201–208

Nan L, Wonka P, (2017) Polyfit: Polygonal surface reconstruction from point clouds. In Proceedings of the IEEE International Conference on Computer Vision 2017, pp. 2353-2361.

Schwab B, Haas-Goschenhofer S and Wysocki O, (2020) LoD3 Road Space Models. https://github.com/savein/lod3-road-space-models (31 January 2021)

Zhu J, Gehrung J, Huang R, Borgmann B, Sun Z, Hoegner L, Hebel M, Xu Y, Stilla U (2020) TUM-MLS-2016: An annotated mobile LiDAR dataset of the TUM city campus for semantic point-cloud interpretation in urban areas. Remote Sensing, 12(11): 1875