Dynamic Sensor Adaptation

My research takes an integrated, cross-layer approach addressing both sensing and processing nodes, hardware and software, aiming to understand and control the interactions between these components for dynamic adaptation. This work was funded by the DARPA Reimagine Program.

Robust Sensor Intelligence Framework

Edge environments like resource constraints or dynamic weather can introduce input corruptions, challenging sensor reliability. While traditional input augmentation assumes ideal noise (e.g., Gaussian), real-world conditions often involve complex patterns like sensor-induced noise or rain. I developed a spatiotemporal rain removal network to detect and restore rain-aggected regions, improving action detection. Additionally, I created a cross-layer simulation methodology incorporating sensor design and process-voltage-temperature (PVT) variations via SPICE simulations during DNN training and evaluation. This work is published in BMVC’19, AICAS’20, and JETCAS’20.

Early Warning-based Adaptive Sensor

While robust deep learning enhances sensor reliability, it cannot eliminate all failures, especially in safety-critical environments. To proactively manage potential disruptions, I developed an adaptive sensor framework that adjusts pixel operations based on early data quality assessments, enhancing resource efficiency without compromising accuracy. This framework leverages model uncertainty as a control metric. Traditional uncertainty estimation methods, like Bayesian neural networks and Monte Carlo sampling, introduce latency; I addressed this with ModelNet, a sampling-free DNN that estimates uncertainty efficiently, achieving up to 179x less computational cost than MC dropout. Additionally, CLUE, a cross-layer uncertainty estimator, accounts for processing hardware variations, reducing calibration error with minimal energy overhead. ModelNet and CLUE together enable an uncertainty-driven sensor control system, enhancing task accuracy and frame rate. This work has been published in DAC’20, DATE’21, Sensors-J’21, IJCNN’22 (Best Paper), and IJCNN’23. Moreover, this research on predicting the trustworthiness of predictions and controlling sensor operations became a foundational concept for the CogniSense center in the JUMP2.0 program.