Embedded Files
Background
Background
AI-generated causal explanations may sound plausible while containing unsupported findings, incorrect anatomical information, invalid causal relationships, or internal contradictions. Such errors can be particularly dangerous in medical applications.
Task 2 evaluates whether an AI system can critically review a causal explanation, identify hallucinations, and produce a safe, evidence-grounded correction.
Task
Task
Participants are given:
• a causal exploration section to be verified;
• the original MIMIC-CXR radiology report; and
• the corresponding MIMIC-CXR chest X-ray image.
The causal exploration section may be valid or may contain one or more controlled hallucinations.
Participants may freely use:
• the report only;
• the image only; or
• both the report and image.
No advance selection of an evidence category is required.
Required Output
Required Output
Each submitted run must provide:
Validity: valid or invalid
Error span: the exact location of each error
Hallucination type: the category of each identified error
Corrected causal explanation
Confidence score from 0 to 1
Each run must also indicate the evidence actually used:
• R — Report-only
• I — Image-only
• RI — Report-and-Image
Important Notes
Important Notes
• Participants must not assume that every explanation contains an error.
• All participants receive the same report and image data.
• R, I, and RI are evidence-use categories, not separate subtasks.
• All Task 2 reports and images are restricted MIMIC-CXR data.
• Every team member who accesses the data must obtain individual PhysioNet authorization.
More Information
More Information
For detailed information, see:
Page updated
Google Sites
Report abuse