Publications

Task: Federated Learning Algorithm Design for Cooperative Perception

Cooperative perception is essential to enhance the efficiency and safety of future transportation systems, requiring extensive data sharing among vehicles on the road, which raises significant privacy concerns. Federated learning offers a promising solution by enabling data privacy-preserving collaborative enhancements in perception, decision-making, and planning among connected and autonomous vehicles (CAVs). However, federated learning is impeded by significant challenges arising from data heterogeneity across diverse clients, potentially diminishing model accuracy and prolonging convergence periods. This study introduces a specialized federated learning framework for C-ITS, termed the federated dynamic weighted aggregation (Fed-DWA) algorithm, facilitated by dynamic adjusting loss (DALoss) function. This framework employs dynamic client weighting to direct model convergence and integrates a novel loss function

that utilizes Kullback-Leibler divergence (KLD) to counteract the detrimental effects of non-independently and identically distributed (Non-IID) and unbalanced data. Utilizing the BEV transformer as the primary model, our rigorous testing on the OpenV2V dataset, augmented with FedBEVT data, demonstrates significant improvements in the average intersection over union (IoU). These results highlight the substantial potential of our federated learning framework to address data heterogeneity challenges in C-ITS, thereby enhancing the accuracy of environmental perception models and facilitating more robust and efficient collaborative learning solutions in the transportation sector.

Task: LiDAR Point Cloud-based Multiple Extended Target Tracking

Multiple target tracking (MTT) is a crucial element in autonomous driving and intelligent transportation systems. Recently, extended target tracking (ETT) has aroused attention in the context of point cloud-based MTT, and several ETT methods based on data-region association (DRA) have been proposed for tracking vehicle targets. The DRA methods can accurately describe the uneven distribution of point clouds by dividing the target extent into separate regions. However, the constrained estimation required by DRA methods increases complexity and approximation errors, while the accuracy and stability for calculating association probabilities are affected by the randomly distributed scattering center. In this paper, we propose a novel ETT algorithm based on probabilistic measurement-region association (PMRA) and Poisson multi-Bernoulli mixture (PMBM) filter for tracking multiple vehicle targets with LiDAR point clouds. The PMRAPMBM algorithm addresses the limitations of DRA model by eliminating the estimation constraint and calculating association probabilities with continuous integrals. Simulation results show the superior estimation accuracy of PMRA-PMBM on target location and extent comparing with the gamma Gaussian inverse Wishart and DRA implementations of the PMBM filter.

Task: Survey of Cooperative Perception with V2X Communication for ADAS and AD

This paper provides a comprehensive overview of the evolution of cooperative perception (CP) technologies, spanning from early explorations to recent developments until Aug. 2023, including advancements in V2X communication technologies. Additionally, we propose a contemporary generic framework to illustrate the V2X cooperation workflow, aiding in the structured understanding of CP system components. Furthermore, this paper categorizes prevailing V2X CP methodologies based on the critical issues they address. We conduct an extensive literature review within this taxonomy, evaluating existing datasets and simulators. Finally, we discuss open challenges and future directions in CP for autonomous driving, considering both perception and V2X communication advancements.

Task: 3D Multi-Object Tracking (MOT) with 4D Imaging Radar

This paper provides the first systematical investigation of the EOT framework for online 3D MOT in real-world ADAS and AD scenarios. Specifically, the widely accepted TBD-POT framework, the recently investigated JDT-EOT framework, and our proposed TBD-EOT framework are compared via extensive evaluations on two open source 4D imaging radar datasets: View-of-Delft and TJ4DRadSet. These provide the first benchmark and important insights for the future development of 4D imaging radar-based online 3D MOT. 

Task: 3D Multi-Object Tracking (MOT) with LiDAR Point Cloud

This paper proposes a learning and graph-optimized (LEGO) modular tracker to improve data association performance in the existing literature. The proposed LEGO tracker integrates graph optimization and self-attention mechanisms, which efficiently formulate the association score map, facilitating the accurate and efficient matching of objects across time frames. LEGO ranked 1st at the time of submitting results to KITTI object tracking evaluation ranking board and remains 2nd at the time of submitting this paper, among all online trackers in the KITTI MOT benchmark for cars. 

Task: 3D Object Detection with Radar and LiDAR Fusion

In this paper, we propose a bird’seye view fusion learning-based anchor box-free object detection system, which fuses the feature derived from the radar rangeazimuth heatmap and the LiDAR point cloud to estimate the possible objects. Different label assignment strategies have been designed to facilitate the consistency between the classification of foreground or background anchor points and the corresponding bounding box regressions. In addition, the performance of the proposed object detector is further enhanced by employing a novel interactive transformer module. The superior performance of the methods proposed in this paper has been demonstrated using the recently published Oxford Radar RobotCar dataset. Our system is compared with the apporaches proposed for the same task with same dataset in CVPR 2020, 2021, 2022, 2023 and ICASSP 2023, significantly outperforms the best one among them by 13.1% and 19.0% at IoU of 0.8 under ’Clear+Foggy’ training conditions for ’Clear’ and ’Foggy’ testing, respectively, thus serves as the latest benchmark for radar and LiDAR fusion-based BEV object detection.

Task: 3D Object Detection with 4D Imaging Radar and Camera Fusion

In this paper, we investigate the “sampling” view transformation strategy on the camera and 4D imaging radar fusion-based 3D object detection. In the proposed LiDAR Excluded Lean (LXL) model, predicted image depth distribution maps and radar 3D occupancy grids are generated from image perspective view (PV) features and radar bird’s eye view (BEV) features, respectively. They are sent to the core of LXL, called “radar occupancy-assisted depth-based sampling”, to aid image view transformation. Introducing image depths and radar information enhances the “sampling” strategy and leads to more accurate view transformation. Experiments on VoD and TJ4DRadSet datasets show that the proposed method outperforms the stateof-the-art 3D object detection methods (e.g., FUTR3D(CVPR 2023), BEVFusion(ICRA 2023), RCFusion(IEEE T-IM), RCN(ICCV 2023) with 4D imaging radar and camera fusion settings) by a significant margin without bells and whistles. Ablation studies demonstrate that our method performs the best among different enhancement settings. This is an early attempt in this field, and serves as the latest benchmark for subsequent studies. 

Task: 3D Object Detection with 4D Imaging Radar

This paper introduces spatial multi-representation fusion (SMURF), a novel approach to 3D object detection using a single 4D imaging radar. SMURF leverages multiple representations of radar detection points, including pillarization and density features of a multidimensional Gaussian mixture distribution through kernel density estimation (KDE). Experimental evaluations on View-of-Delft (VoD) and TJ4DRadSet datasets demonstrate the effectiveness and generalization ability of SMURF, outperforming recently proposed 4D imaging radarbased single-representation models. Moreover, while using 4D imaging radar only, SMURF still achieves comparable performance to the state-of-the-art 4D imaging radar and camera fusion-based method, with an increase of 1.22% in the mean average precision on bird’s-eye view of TJ4DRadSet dataset and 1.32% in the 3D mean average precision on the entire annotated area of VoD dataset. This research highlights the benefits of 4D mmWave radar and is a latest benchmark for subsequent works regarding 3D object detection with 4D imaging radar. 

Task: 3D Multi-Object Tracking (MOT) with LiDAR

In this paper, it is demonstrated that the latest RFS-based Bayesian tracking framework could be superior to typical random vector-based Bayesian tracking framework via a systematic comparative study of both traditional random vector-based Bayesian filters with rule-based heuristic track maintenance and RFS-based Bayesian filters on the nuScenes validation dataset. An RFS-based tracker, namely Poisson multiBernoulli filter using the global nearest neighbor (GNN-PMB), is proposed to LiDAR-based MOT tasks. This GNN-PMB tracker is simple to use, and it achieves competitive results on the nuScenes dataset. Specifically, the proposed GNN-PMB tracker outperforms most state-of-the-art LiDAR-only trackers and LiDAR and camera fusion-based trackers, ranking the 3rd among all LiDAR-only trackers on nuScenes 3D tracking challenge leader board at the time of submission. 

Task: Instance Segmentation with Automotive Radar with Contrastive Learning

To solve the issue that high-quality annotations of radar detection points are challenging to achieve due to their ambiguity and sparsity, we propose a contrastive learning approach for implementing radar detection points-based instance segmentation. Experiments show that when the ground-truth information is only available for a small proportion of the training data, our method still achieves a comparable performance to the approach trained in a supervised manner with 100% ground-truth information.

Task: Instance Segmentation with Automotive Radar

In this paper, we propose an efficient method based on clustering of estimated semantic information to achieve instance segmentation for the sparse radar detection points. In addition, we show that the performance of the proposed approach can be further enhanced by incorporating the visual multi-layer perceptron. The effectiveness of the proposed method is verified by experimental results on the popular RadarScenes dataset, achieving 89.53% mean coverage and 86.97% mean average precision with the IoU threshold of 0.5, which is superior to other approaches in the literature. More significantly, the consumed memory is around 1MB, and the inference time is less than 40ms, indicating that our proposed algorithm is storage and time efficient. These two criteria ensure the practicality of the proposed method in real-world systems. 

Task: Unsupervised Learning MRI Super Resolution with Limited HR MRI Images

Magnetic resonance imaging (MRI) is crucial for enhancing diagnostic accuracy in clinical settings. However, the inherent long scan time of MRI restricts its widespread applicability. Deep learning-based image super-resolution (SR) methods exhibit promise in improving MRI resolution without additional cost. Due to lacking of aligned high-resolution (HR) and low-resolution (LR) MRI image pairs, unsupervised approaches are widely adopted for SR reconstruction with unpaired MRI images. However, these methods still require a substantial number of HR MRI images for training, which can be difficult to acquire. To this end, we propose an unpaired MRI SR approach that employs contrastive learning to enhance SR performance with limited HR training data. Empirical results presented in this study underscore significant enhancements in the peak signal-to-noise ratio and structural similarity index, even when a paucity of HR images is available. These findings accentuate the potential of our approach in addressing the challenge of limited HR training data, thereby contributing to the advancement of MRI in clinical applications. 

Task: Supervised Learning MRI Parallel Reconstruction with Implicit Neural Representation

In this paper, we propose a novel MRI reconstruction method based on INR, which represents the fully-sampled images as the function of pixel coordinates and prior feature vectors of undersampled images for overcoming the generalization problem of INR. Specifically, we introduce a scale-embedded encoder to produce scale-independent pixel-specific features from MR images with different undersampled scales and then concatenate with coordinates vectors to recover fully-sampled MR images via an MLP, thus achieving arbitrary scale reconstruction. The performance of the proposed method was assessed by experimenting on publicly available MRI datasets and compared with other reconstruction methods. Our quantitative evaluation demonstrates the superiority of the proposed method over alternative reconstruction methods. 

Task: Unsupervised Learning MRI Super Resolution

Training neural networks for MRI super resolution application in supervised learning strategy requires aligned authentic HR and LR image pairs, which are challenging to obtain due to patient movements during and between image acquisitions. While rigid movements of hard tissues can be corrected with image registration, aligning deformed soft tissues is complex, making it impractical to train neural networks with authentic HR and LR image pairs. Previous studies have focused on SRR using authentic HR images and down-sampled synthetic LR images. However, the difference in degradation representations between synthetic and authentic LR images suppresses the quality of SR images reconstructed from authentic LR images. To address this issue, we propose a novel Unsupervised Degradation Adaptation Network (UDEAN). Our network consists of a degradation learning network and an SRR network. The degradation learning network downsamples the HR images using the degradation representation learned from the misaligned or unpaired LR images. The SRR network then learns the mapping from the down-sampled HR images to the original ones. Experimental results show that our method outperforms state-of-the-art networks and is a promising solution to the challenges in clinical settings. 

Task: Supervised Learning MRI Super Resolution with Uncertainty Estimating

We adopted a unified framework of 2D deep learning neural network for both 3D MRI super-resolution and motion artifact reduction, demonstrating such a framework can achieve better performance in 3D MRI restoration tasks compared to other stateof-the-art methods and remain the GPU consumption and inference time significantly low, thus easier to deploy. We also analyzed several down-sampling strategies based on the acceleration factor, including multiple combinations of in-plane and through-plane down-sampling, and developed a controllable and quantifiable motion artifact generation method. At last, the pixel-wise uncertainty was calculated and used to estimate the accuracy of the generated image, providing additional information for a reliable diagnosis.