Human Behavior and Cognitive States Prediction (Pre-Event)

To provide drivers with safer Physical AI systems, it is essential to understand and predict what actions they will take and what cognitive states they will experience seconds or minutes in advance. By accurately forecasting these future behaviors and mental states, Physical AI systems can deliver timely assistance and proactive safety interventions.

My research leverages multimodal driver data to predict critical outcomes such as driver fatigue, impending collisions, and hazard responses within the driving domain:

• Accident Prediction: Developed machine learning models using driver control metrics (pedal inputs and steering wheel dynamics) to predict pedestrian collisions 2.5 seconds prior to the potential accident [J3].

• Obstacle Type Detection: Built machine learning models integrating physiological signals and vehicle telemetry to classify which type of obstacle a driver is responding to 6 seconds before interaction [J8].

• Fatigue Detection: Investigating real-time driver fatigue during automated driving by combining multimodal data, including eye tracking, heart rate, galvanic skin response (GSR), and 3D skeleton motion capture, to achieve faster and more accurate fatigue detection than conventional models [Working Paper].