Research

I coordinate the ETH Zurich part of an industry-academy collaboration with Align Technology and the University of Geneva. The goal of this research project is the learning-based simulation and the high-quality rendering of soft-tissue facial changes due to orthodontic treatments. This will allow the patients to previsualize the changes in their faces due to the treatment, contributing to facilitated patient-doctor communication, more accurate treatment outcomes, and increased patient satisfaction. We employ highly heterogeneous medical data consisting of computed tomography (CT), intraoral and facial scan modalities and develop a digital model of the patient, which is then used for simulation of the treatment and photorealistic rendering.

Cleft lip and palate is a birth defect with a 1 in 700 frequency, affecting approximately 200,000 infants worldwide every year. As part of the BRCCH-funded multidisciplinary project Burden-Reduced Cleft Lip and Palate Care and Healing, of which the technical part I coordinate at ETH Zurich supervising a PhD student and several bachelor students, I conceived and led the development of a fully automated and digital method for computing presurgical plates for infants with cleft lip and palate. The plates facilitate the feeding of the infants and provide a closure of the cleft over the course of a few months, which in return benefits the surgical outcome. The traditional method of generating the plates with silicon impression is cumbersome, requires a dental technician, and puts the airway of infants at risk. Our method requires only an intraoral scan of the infant, and for the training of the machine learning algorithms, it employs a large 3D intraoral scan dataset provided by clinical partners in Switzerland, Poland, and India. The automatically computed plates are then fabricated using 3D printing. Our solution and its graphical user interface (GUI) became part of the clinical routine at the University Hospital of Basel. It has been used for the treatment of more than 30 patients in Switzerland and India (Saveetha Medical Hospital, Chennai, and Bhagwan Mahaveer Jain Hospital, Bangalore). The successful clinical translation of our work was recently awarded by Best Bench-to-Bedside Award at IPCAI Conference in Munich in June 2023.  

A related project I am supervising is in the field of computer vision and artificial intelligence, and its goal is to generate 3D intraoral scans of infants based on smartphone camera photos and videos. Since intraoral scanners are expensive and smartphones are endowed with increasingly more accurate sensors, this could allow patients in low-income countries to be scanned cost-effectively. This can then be used for dental treatments or presurgical plate computation for cleft lip and palate. So far, we have attained submillimeter accuracy in 3D reconstruction in this challenging part of the body, and it has also been tested in a surgical setting at the University Hospital of Basel. For the presurgical plates, we have previously used intraoral scans as input, but in July 2023, three patients in Hyderabad, India, started treatment with automatically computed plates generated based on smartphone-based 3D palatal reconstruction. 

The existing facial models are almost exclusively generated from adult datasets, rendering their use impossible on applications with infants since infants have different facial and head shapes. In this collaboration with Prof. Andreas Müller’s Research Group Facial & Cranial Anomalies at the University of Basel, we develop face and head models specifically for infants. These models will be used for the treatments of common craniofacial malformations of children. We aim at patient modelling, visualization, and simulations for the treatment of these malformations. 

Magnetic resonance imaging (MRI) scans are costly. Moreover, for claustrophobic people, children, and patients who cannot hold their breaths, the long scanning times constitute a challenge. Therefore, it’s high of interest to enable faster scans. When I started my PhD, there were already image processing techniques to reconstruct MRI scans from less Fourier samples instead of the full sampling. However, I had observed back then that there was no systematic method of choosing the sampling schemes, i.e., which samples to take during the scan, except for heuristical solutions. Therefore, I proposed this problem as the focus of PhD thesis proposal and developed a learning-based sampling optimization for MRI. My research used training data acquired from clinical partners such as Lausanne University Hospital (CHUV). Our sampling optimization algorithms allowed higher-quality image reconstructions tailored to the anatomy, the preferred image quality metric, and classical or deep learning-based reconstruction algorithms for a given scan acceleration factor. Our publication in IEEE Transactions on Medical Imaging and the follow-up work addressing the multi-coil and dynamic MRI settings triggered a substantial interest among the medical imaging community in the topic; many research groups followed up on the idea, investigating the joint optimization of sampling and reconstruction algorithms or reinforcement learning for online sampling decisions.