Overview - Join Us – Register Here

Live commentary plays a vital role in helping audiences interpret complex, high-stakes events such as political debates, Federal Reserve press conferences, and corporate earnings calls. Unlike simple summarization, expert commentary involves timely decisions about what to comment on and how to present it — integrating fact-checking, background knowledge, and subjective evaluation. Our project introduces the first multi-domain dataset for studying Live Commentary Planning and Generation, aligning event transcripts with time-synchronized expert analyses and public reactions.

Research Motivation

While large language models can generate fluent text, true expert commentary requires:

• Strategic selection of key moments to analyze

• Integration of external knowledge and fact-checking

• Balanced subjective evaluation and explanation

Despite growing interest in debate analysis, financial communication, and news summarization, no existing dataset aligns live transcripts with expert commentary. We aim to bridge that gap and enable research on AI systems that can plan and write commentary like professionals.

Tasks

Commentary Planning

Predict the type of commentary an expert would make for a given transcript segment.

• Task: Multi-class classification (e.g., 11-way for debates)

• Metrics: Accuracy, macro/micro F1

Commentary Generation

Generate the text of the commentary, given:

• A transcript segment

• A target commentary label

Evaluated with:

• Automatic metrics: ROUGE, BERTScore

• Human judgments: Importance, Novelty, Clarity, and Factual Accuracy

Dataset Details- refer to our paper for more details

Our dataset brings together expert and public live commentary from three major domains: U.S. presidential debates, Federal Open Market Committee (FOMC) press conferences, and corporate earnings calls. In addition, we also include public reactions from Reddit discussions during presidential debates to represent non-expert perspectives.

For the U.S. presidential debates, we collected transcripts from the 2016, 2020, and 2024 election cycles, along with Bloomberg’s real-time expert commentary. After careful alignment by timestamp, the debate dataset contains 2,283 pairs of transcript segments and commentary texts, annotated with 11 distinct commentary categories.

For the FOMC press conferences, we used Bloomberg’s live coverage of the Federal Reserve Chair’s statements and Q&A sessions. This subset contains 252 aligned pairs with 5 commentary categories that reflect financial sentiment and interpretation.

For the corporate earnings calls, we focused on S&P 500 companies across multiple sectors. Each call includes a management presentation and analyst Q&A, accompanied by Bloomberg’s expert financial commentary. This portion of the dataset contains 1,115 aligned pairs labeled with 10 detailed categories, covering summaries, comparisons, opinions, and analytical remarks.

Finally, for the Reddit debate commentary, we incorporated public reactions from Reddit threads created during the 2016 debates. Using topic-based matching, we aligned 366 Reddit comments to specific debate utterances and categorized them into four general types — agreement, disagreement, elaboration, and paraphrase — representing how ordinary viewers respond to debate content in real time.

In total, the dataset contains over 3,600 expert commentary instances, plus hundreds of crowd-sourced comments, making it the most comprehensive resource for studying real-time commentary across political, financial, and public domains.

Schedule

Below is the tentative timeline for the Live Commentary Planning and Generation shared task. All dates are in 2025–2026, and may be subject to minor adjustments.

2025

• December 1st, 2025 — Release of the Training Set

2026

• April 15th, 2026 — Release of the Test Set

• May 1st, 2026 — System Output Submission Deadline

• June 1st, 2026 — Evaluation Results and Leaderboard Announcement

• July 1st, 2026 — Paper Submission Deadline

• August 1st, 2026 — Notification of Acceptance

• August 15th, 2026 — Camera-Ready Paper Deadline

• October–November 2026 (TBD) — INLG 2026 Conference

Organizers - Contact us: 

• Chung-Chi Chen (AIST, Japan)

• Huan-Wen Ho (National Chung Cheng University, Taiwan)

• Yu-Yu Chang (National Chung Cheng University, Taiwan)

• Ming-Hung Wang (National Chung Cheng University, Taiwan)

• Ramon Ruiz-Dolz  (University of Dundee, UK)

• Chris Reed (University of Dundee, UK)

• Ichiro Kobayashi (Ochanomizu University, Japan)

• Yusuke Miyao (University of Tokyo, Japan)

• Hiroya Takamura (AIST, Japan)