In this section, we discuss the challenges in our experiments and opportunities for future work.
According to our findings from RQ1-3, existing AI-enabled MSF systems are not robust enough. First, corrupted signals could result in significant performance degradation of AI-enabled MSF systems. The data-driven nature makes it challenging to train a robust MSF system for all conditions. Therefore, more research on the continuous enhancement of AI-enabled MSF is needed, such as debugging and repair. In addition, fusing more types of signals (e.g., radar, which has longer wavelengths and performs better in rainy and foggy environments) could be another potential way to improve robustness.
Our findings from RQ2 also reveal that AI-enabled MSF systems are sensitive to calibration and synchronization errors. In the real world, these two types of errors always exist. Even well-calibrated sensors can still be misaligned due to the changes in external environments. To deploy a reliable AI-enabled MSF system, developers must address the calibration issues carefully.
Modular redundancy is a critical way to improve system quality and reliability. By coupling multiple sensors, AI-enabled MSF systems are expected to be robust against signal loss from one specific sensor. However, our experimental results suggest that existing work usually ignores this when designing AI-enabled MSF, resulting in a lack of robustness.
While even existing AI-enabled MSF systems are not robust enough, it is possible to fix them with improved fusion mechanisms. In this work, we propose improved weak and late fusion mechanisms. The experimental results demonstrate their effectiveness, showing that improving fusion mechanisms could be a promising research direction.
Based on these insights, we summarize the following future directions:
We initiate to create an early public benchmark of AI-enabled MSF-based perception systems, which provides a common ground for evaluating AI-enabled MSF systems' robustness and enables the researchers and practitioners to perform systematic study and research. However, MSF can also be used in systems beyond perception and autonomous driving. We encourage more research efforts on benchmarks and empirical studies in this direction and more fine-grained robustness evaluation metrics for AI-enabled MSF systems should be considered in the future. This is beneficial to understand the exist robustness issues or risks when deploying AI-enabled MSF systems in the real world.
There is an urgent need for robustness enhancement techniques to continuously improve the reliability of AI-enabled MSF systems. However, To our knowledge, few methods has been proposed to test and enhance the AI-enabled MSF-based perception systems. Based on our investigation results, improving fusion mechanisms to repair MSF systems could be a promising research direction.
Moreover, the fusion mechanism reflects the inherent characteristics of the MSF system. Therefore, more research efforts on improving fusion mechanisms are necessary. Designing more robust and advanced fusion mechanisms could be critical for MSF systems to work well in the real world. Besides, different fusion mechanism-based MSF systems show different robustness issues. Therefore, practical software and system engineering approaches (e.g., testing, debugging, formal analysis, and repairing) would be needed for different MSF systems.