Research Statement

My general research interests lie in automated visual data collection, processing, and interpretation for enabling more efficient and effective construction/design integration and construction/infrastructure management. As noticed by many in the industry, more and more civil engineers have begun to utilize visual data, such as photos, videos and laser scanned point clouds in their daily work. Rich and comprehensive visual information captured in these data sources, such as spatial relationships between building components, geometric and color information of a facility, have a wide range of applications. Designers need as-is geometric and color information of a building for a renovation design. Constructors need frequent 3D snapshots of a construction site for equipment tracking, safety management, construction progress monitoring, and construction tolerance analysis. Facility managers need the deformation history and crack information of a facility for regular condition assessments and preventive maintenance planning.

As the costs of visual sensors decrease and the quality of their data increase, it becomes critical that how to efficiently and effectively utilize those visual sensors and large amounts of data generated by them. Current approaches of utilizing visual data in practice involve large amounts of subjective and tedious manual work of civil engineers, which can potentially cause inefficiencies, errors, and delays of building and construction projects. For instance, if an inspector need to check whether all constructed columns are vertical using a 3D imaging system, which can capture accurate 3D as-built conditions, he/she need to manually identify locations for taking 3D images, and ensure that all columns are captured in these 3D images. For a construction site with many columns scattering around and many objects irrelevant for column quality control (e.g. other building components, equipments), manual identification of a set of locations seeing all columns is time-consuming and error-prone. Moreover, retrieving geometric information on all columns from the 3D data requires civil engineers to manually extract geometric features for each column and determine its verticalness. That data interpretation procedure involves large amounts of repetitive manual operations, and any improper operation during that process can cause incorrect evaluation results about column conditions. In my opinion, it is crucial to enable automatic executions of those repetitive and tedious manual operations to streamline the utilization of visual data in the civil engineering domain.

The vision of automated visual data collection/processing/interpretation is closely related to the integration of the design and construction. Building Information Modeling (BIM) enables civil engineers to describe their designs in terms of building components, spaces, and relationships between those objects, but a design model might not necessarily reflect the actual conditions of a constructed facility. Algorithms developed in the domains of Remote Sensing, Computer Vision, and Computational Geometry are capable of recovering object-oriented semantic information from raw data, but manually execute these algorithms to extract as-built geometric information and update a BIM is tedious. Integration of a BIM with visual data makes it possible to augment data with the semantic information from the BIM for intelligent as-built information query and data interpretation. On the other hand, automatically extracted as-built information can be used to update the BIM. That automatic bi-direction as-designed/as-built information flow can streamline the interactions between designers and constructors. My future research projects will focus on how to utilize a BIM to guide effective visual data collection, how to automatically update a BIM with collected visual data, and how to use a BIM augmented by visual data to answer queries from civil engineers.

In the last four years, I have conducted extensive studies about how to streamline the utilization of 3D laser scanning technology in the domains of construction quality control and bridge inspection. First, aiming at enabling efficient automated laser scanner configuration for detecting defects of constructed facilities, I conducted a series of case studies using three different scanners on construction sites. In these case studies, I analyzed how civil engineers manually decide scanner configurations, and how those decisions influence the data quality, coverage of information goals, and the data collection time. I have developed a sensor model simulating the correlations between the sensor-configuration and the data quality. In my future research, I plan to continue this work and develop more general sensor models for various visual sensors. With those visual sensor models, it is possible to develop reasoning algorithms, which can automatically suggest proper sensor settings given the job site condition, sensor information, and information requirements of civil engineers.

Second, aiming at enabling automated 3D data processing, I investigated how civil engineers select and configure data processing algorithms for various tasks (e.g. removal of data noise), and how algorithm configurations impact the resulted data quality. My comparative analysis of the performances of many data processing algorithms indicates that for different data sets, the best algorithm might vary, and some deterministic correlations exist between algorithm parameter values and the quality of the processed data. In my future research, I plan to develop algorithm performance models and reasoning mechanisms to automatically identify proper algorithm settings for given data processing tasks of civil engineers.

Third, aiming at enabling automated 3D data interpretation for streamlining the 3D-data-driven decision-making process of civil engineers, I have developed computer interpretable representations of geometric information goals and relevant reasoning mechanisms for automated extraction of geometric information goals from 3D data. This work targets enabling civil engineers to define geometric information goals (e.g. column cross section area) using a computer interpretable format, and then the developed reasoning mechanisms can automatically generate and execute data processing workflows by reasoning about that user-defined information goal. This approach frees civil engineers from repetitive manual data processing tasks, and enables them to interpret more visual data in limited time. In my future research, I plan to augment this approach with machine learning techniques to enable computers to capture and learn visual information retrieval workflows from experienced civil engineers during the human-computer interaction process.

Future civil engineers will manage increasingly large infrastructure systems and complicated construction sites. They need comprehensive, faithful, accurate and up-to-date snapshots of their job sites and facilities for making proper decisions. A BIM augmented by visual data can satisfy many of these requirements. By freeing civil engineers from manual operations during the visual data collection/processing/interpretation workflow, my research can help engineers to get more out of these data, accelerate the decision-making procedure, and reduce costive errors. The funding supporting my current research projects is from a variety of agencies interested in various applications of such techniques. NSF and Bombardier Transportation fund my research about 3D data accuracy analysis and the application of 3D data for automated defect detection on construction sites. General Services Administration supports our research about quality assessment of 3D as-built models for building renovation and operation. NIST supports our research about automatic integration of point clouds with building information models for enabling semantically rich queries of as-built information for facility management. PennDOT is interested in how visual data can improve their bridge visual inspection practice. Having working closely with those funding agencies, I am aware of the wide audiences of my research, and plan to write proposals attracting their interests to fund my future explorations in this exciting area.

If you are from an institution which has strong interests in automated management of construction, facility and infrastructure, I believe I can bring my knowledge about the performance models of sensors/algorithms and automated data processing/interpretation to your group. I am looking forward work together with you to make visual data more accessible to construction managers, facility managers and infrastructure management agencies, so that they can make more rational decisions based on more reliable information from the data.