This year we have partnered with collaborators at Sony and EPFL to be the official host for the two tasks for Peacock Shared Task.
Total number of submissions: 1527 (task 1), 1259 (task 2)
Total number of participating teams: 38 (task 1), 18 (task 2)
Challenge Summary Paper: ComperDial: Commonsense Persona-grounded Dialogue Dataset and Benchmark (Evaluation Dataset)
Paper submission for winning entries was optional.
1st Prize: Team ni_kai_hua_, Paper, Slide deck
2nd Prize: Team wangzhiyu918
3rd Prize: Team justsnail
1st Prize: Team biu_biu_team, Paper, Slide deck
2nd Prize: Team test_team
3rd Prize: Team TieMoJi, Paper, Slide deck
To sustain coherent and engaging conversations, dialogue agents must consider the personas of listeners to produce utterances that cater to their interests. They must also maintain consistent speaker personas, so that their counterparts feel engaged in a realistic conversation. However, achieving this persona alignment requires understanding the inter-connected interests, habits, experiences, and relationships of various real-word personas, and leveraging them effectively to robustly and believably engage in conversations. In this task, we are calling for dialogue response generation systems that can make good representation and incorporation of personas grounded on commonsense. We prepared a new persona-grounded dialogue dataset for the evaluation in open-domain dialogue, which was created based on a Persona Commonsense Knowledge (hereinafter PeaCok) Graph released at ACL2023. Each dialogue is a conversation between a pair of workers assigned a persona. Each persona consisted of several profile sentences including some personal information (e.g. name, age, etc.), head personas in PeaCok, and tail personas corresponding to the head personas picked out from PeaCok. The dialogues are consistent throughout because profile sentences are organized naturally related to each other based on the Knowledge Graph.
For details please refer to this page: Task 1 webpage. This is the link for the challenges Discord channel.
The challenge will take place across 3 Rounds which differ in the evaluation dataset used for ranking the systems.
Warm-up Round: 3rd November, 2023
Round 1: 8th December, 2023 (23:59 UTC)
Round 2: 15th January, 2024 (23:59 UTC)
Team Freeze Deadline: 24th February, 2023 (23:59 UTC)
Challenge End: 1st March, 2024 (23:59 UTC)
🥇 First place 15,000 USD
🥈 Second place 7,000 USD
🥉 Third place 3,000 USD
Understanding rich dialogues often requires NLP systems to access relevant commonsense persona knowledge that grounds the dialogue. However, it is challenging to retrieve such relevant persona commonsense from knowledge bases, due to the complex contexts of real-world dialogues, and the implicit and ambiguous nature of commonsense knowledge. In this task, we are calling for commonsense persona knowledge linkers that can robustly identify relevant commonsense facts associated with speakers and listeners in dialogues.
For details please refer to this page: Task 2 webpage. This is the link for the challenges Discord channel.
The challenge will take place across 3 Rounds which differ in the evaluation dataset used for ranking the systems.
Warm-up Round: 3rd November, 2023
Round 1: 8th December, 2023 (23:59 UTC)
Round 2: 15th January, 2024 (23:59 UTC)
Team Freeze Deadline: 24th February, 2023 (23:59 UTC)
Challenge End: 1st March, 2024 (23:59 UTC)
🥇 First place 5,000 USD
🥈 Second place 3,000 USD
🥉 Third place 2,000 USD
If you have questions regarding shared tasks please contact us: nlp4convai-organizers@googlegroups.com
This is the link for the challenges Discord channel.