Task 2:
Causal Verification and Correction

Background

AI-generated causal explanations may sound plausible while containing unsupported findings, incorrect anatomical information, invalid causal relationships, or internal contradictions. Such errors can be particularly dangerous in medical applications.

Task 2 evaluates whether an AI system can critically review a causal explanation, identify hallucinations, and produce a safe, evidence-grounded correction.

Task Definition

Participants are given:

• a causal exploration section to be verified;

• a link to the original radiology report; and

• a link to the corresponding chest X-ray image.

The causal exploration section may be valid or may contain one or more controlled hallucinations.

Participants may freely use:

• the report only;

• the image only; or

• both as supporting evidence. 

No advance selection of an evidence category is required.

Required Output

Each submitted run must provide:

1. Validity: valid or invalid

2. Error span: the exact location of each error

3. Hallucination type: the category of each identified error

4. Corrected causal explanation

5. Confidence score from 0 to 1

Each run must also indicate the evidence actually used:

• R — Report-only

• I — Image-only

• RI — Report-and-Image

Important Notes

• Participants must not assume that every explanation contains an error.

• All participants receive the same links to the corresponding report and image data.

• R, I, and RI are evidence-use categories, not separate subtasks.

• All Task 2 reports and images are restricted MIMIC-CXR data.

• Every team member who accesses the data must obtain individual PhysioNet authorization.

More Information

For detailed information, see:

• Task 2 Details

• Submission Format

• Hallucination Taxonomy

• MIMIC-CXR Data Access

• Evaluation