Overview

Task 1 requires participants to generate a structured A1–A5 interpretation for each chest X-ray image.

Because the output consists of short structured answers rather than a free-text causal explanation, Task 1 will be evaluated using exact matching and set-based classification metrics.

Each A1–A5 component contributes 20 points to the final score.

A1 — Initial Impressions

A1 may contain multiple initial impressions.

Predictions will be evaluated using:

• precision;

• recall; and

• set-based F1.

Equivalent terms will be mapped to a canonical terminology before scoring.

A2 — Thoracic Level or Lung Zone

A2 will be evaluated using normalized exact match when a single answer is expected.

When multiple answers are permitted, set-based precision, recall, and F1 will be used.

A3 — Anatomical Location

A3 will be evaluated using normalized exact match or set-based F1.

The evaluation will consider:

• anatomical region;

• lung zone or compartment, when applicable; and

• laterality.

A4 — Final Impressions

A4 may contain one or more final impressions.

Predictions will be evaluated using:

• precision;

• recall; and

• set-based F1.

Normal cases must be identified using the official normal label specified in the submission format.

A5 — Confirmation Checklist

A5 is a multi-label confirmation checklist.

Evaluation will use:

• micro-F1, which reflects overall checklist-item performance; and

• macro-F1, which gives equal importance to frequent and infrequent checklist items.

The A5 score will be calculated from both micro-F1 and macro-F1.

Structured Consistency

When thoracic level and anatomical location are associated with a particular finding or impression, the evaluation may use structured tuples such as:

• impression;

• thoracic level or lung zone; and

• anatomical location.

A system should not receive full credit when it predicts the correct items but connects them incorrectly.

Terminology Normalization

Before evaluation, predictions may be normalized for:

• capitalization;

• leading and trailing spaces;

• punctuation;

• approved abbreviations;

• singular and plural forms; and

• officially recognized synonyms.

The official vocabulary and normalization mapping will be released with the evaluation script.

Task 1 Final Score

• A1 — 20 points

• A2 — 20 points

• A3 — 20 points

• A4 — 20 points

• A5 — 20 points

Total: 100 points