Program

Abstract:

I believe a fully reliable system could be overly trusted.

Trust in an automation is not just a subjective reliability of the automation in a probabilistic meaning. Every machine has a set of conditions within which the machine is supposed to work as the designer intends. Along with the automated driving context, such set of conditions could be called Operational Design Domain (ODD). It is true an automation may make a mistake even within the ODD. You can call it overtrust if (s)he relied on the automation that made a mistake within the ODD. However, the only solution in this case is to improve the quality of the automation. Relying on the automation within the ODD is inherently not a problem. Rather it could be said the automation must be fully reliable within the ODD. On the other hand, people may rely on an automation even if the current situation is beyond the ODD, i.e., the automation cannot work well in the situation. In this sense, the proposition “a human may overtrust in a fully reliable system” could be true if the automation works perfectly within the ODD.

About 20 years ago, I tried to develop a conceptual model to discuss this issue. In this talk, I would like to try it again to obtain deep insights of overtrust in automation.

Abstract:

Properly calibrated human trust is essential for successful interaction between humans and automation. However, while human trust calibration can be improved by increased automation transparency, too much transparency can overwhelm human workload. To address this tradeoff, we present a probabilistic framework using a partially observable Markov decision process (POMDP) for modeling the coupled trust-workload dynamics of human behavior in an action-automation context. We specifically consider hands-off Level 2 driving automation in a city environment involving multiple intersections where the human chooses whether or not to rely on the automation. We consider automation reliability, automation transparency, and scene complexity, along with human reliance and eye-gaze behavior, to model the dynamics of human trust and workload. We demonstrate that our model framework can appropriately vary automation transparency based on real-time human trust and workload belief estimates to achieve trust calibration.

Abstract:

To maximize safety and driving comfort, autonomous driving systems can benefit from implementing foresighted action choices that take different potential scenario developments into account. While artificial scene prediction methods are making fast progress, an attentive human driver may still be able to identify relevant contextual features which are not adequately considered by the system or for which the

human driver may have a lack of trust e into the system’s capabilities to treat them appropriately. We implement an approach that lets a human driver quickly and intuitively supplement scene predictions to an autonomous driving system by gaze. We illustrate the feasibility of this approach in an existing autonomous driving system running a variety of scenarios in a simulator. Furthermore, a Graphical User Interface (GUI) was designed and integrated to enhance the trust and explainability of the system. The utilization of such cooperatively augmented scenario predictions has the potential to improve a system’s foresighted driving abilities and make autonomous driving more trustable, comfortable and personalized

Abstract:

The Thinking Wave is an ongoing development of visualization concepts showing the real-time effort and confidence of semi-autonomous vehicle (AV) systems. Offering drivers access to this information can inform their decision making, and enable them to handle the situation accordingly and takeover when necessary. Two different visualizations have been designed: Concept one “Tidal” demonstrates the AV system’s effort through intensified activity of a simple graphic which fluctuates in speed and frequency. Concept two “Tandem” displays the effort of the AV system as well as the handling dynamic and shared responsibility between the driver and the vehicle system. Working collaboratively with mobility research teams at the University of Tokyo, we are prototyping and refining the Thinking Wave and its embodiments as we work towards building a testable version integrated into a driving simulator. The development of the thinking wave

aims to calibrate trust by increasing the driver’s knowledge and understanding of vehicle handling capacity. By enabling transparent communication of the AV system’s capacity, we hope to empower AV-skeptic drivers and keep over-trusting drivers on alert in the case of an emergency takeover situation, in order to create a safer autonomous driving experience.