Artificial intelligence (AI) has already become a transformative technology that is having revolutionary impact in nearly every domain from business operations to more challenging contexts such as civil infrastructure, healthcare and military defense.  AI systems built on large language and foundational/multi-modal models (LLFMs) have proven their value in all aspects of human society, rapidly transforming traditional robotics and computational systems into intelligent systems with emergent, beneficial and even unanticipated behaviors.  However, the rapid embrace of AI-based critical systems introduces new dimensions of errors that induce increased levels of risk, limiting trustworthiness.  Furthermore, the design of AI-based critical systems requires proving their trustworthiness. Thus, AI-based critical systems must be assessed across many dimensions by different parties (researchers, developers, regulators, customers, insurance companies, end-users, etc.) for different reasons. 

We can call it AI testing, validation, monitoring, assurance, or auditing, but the fundamental concept in all cases is to make sure the AI is performing well within its operational design and avoids unanticipated behaviors and unintended consequences. Such assessment begins from the early stages of research, development, analysis, design, and deployment. Thus, trustworthy AI systems and methods for their assessment should address full system-level functions as well as individual AI-models and require a systematic design both during training and development phases, ultimately providing assurance guarantees.  At the theoretical and foundational level, such methods must go beyond explainability to deliver uncertainty estimations and formalisms that can bound the limits of the AI; find blind spots and edge-cases; and incorporate testing for unintended use-cases, such as adversarial testing and red teaming in order to provide traceability, and quantify risk. This level of performance is critically important to contexts that have highly risk-averse mandates such as, healthcare, essential civil systems including power and communications, military defense, and robotics that interface directly with the physical world.

The symposium track aims to create a platform for discussions and explorations that are expected to ultimately contribute to the development of innovative solutions for quantitatively trustworthy AI. The symposium track will last 2.5 days and will feature keynote and invited talks from accomplished experts in the field of Trustworthy AI, panel sessions, the presentation of selected papers, student papers and a poster session. Potential topics of interest include, but are not limited to:

• Assessment of non-functional requirements such as explainability, including transparency, accountability, and privacy.

• Methods that use data and knowledge to support system reliability requirements, quantify uncertainty, or balance over-generalizability.

• Approaches for verification and validation (V&V)  of AI systems and quantitative AI and system performance indicators.

• Methods and approaches for enhancing reasoning in LLFMs, e.g. causal reasoning techniques and outcome verification approaches.

• Links between performance, and trustworthiness and trust leveraged by AI sciences, system and software engineering, metrology, and Social Sciences and Humanities methods.

• Research on and architectures/frameworks for Mixture-Of-Experts (MoE) and multi-agent systems with an emphasis on robustness, reliability, and emergent behaviors in risk-averse contexts.

• Evaluation of AI systems vulnerabilities, risks and impact; including adversarial (prompt injection, data poisoning, etc.) and red-teaming approaches targeting LLFMs or multi-agent behaviors.

For more information on topics, see our call for papers here.