AffCon2020 - CL-Aff Shared Task

The AAAI-20 Workshop On Affective Content Analysis

AFFCON2020: Interactive Affective Response

February 7, 2020

Room Details: Bryant, 2nd Floor

Hilton New York Midtown

New York, USA

CL-Aff Shared Task 2020: Get it #OffMyChest

There is a growing interest in understanding how humans initiate and hold conversations. The affective understanding of conversations focuses on the problem of how speakers use emotions to react to a situation and to each other. We introduce the OffMyChest conversation dataset, and invite submissions for the Computational Linguistics Affect Understanding (CL-Aff) Shared Task on modeling interactive affective responses.

Tasks

GIVEN: *English* sentences sampled out of the casual and confessional conversations among Redittors on the /r/CasualConversations and the /r/OffMyChest community, labeled for their informational, social and affective characteristics.

TASK 1: Semi-supervised learning task: Predict labels for Disclosure and Supportiveness for sentences based on a small labeled and large unlabeled training data.

TASK 2: Unsupervised task: Propose new characterizations and insights to model conversation dynamics.

Corpus details

Unlabeled training set (Released!):
- UNLABELED POSTS: The top posts in 2018 in /r/CasualConversations and /r/OffMyChest mentionining any of the keyword terms. Posts that are parents of comments in the training and test sets are separately identified.
- UNLABELED COMMENTS: Over 420k sentences extracted from 130k comments posted to "POSTS"
Labeled training set (Released!):
- About 10,000 labeled sentences , extracted from the top comments posted to "POSTS".
- For label descriptions, please check out the annotation instructions under /docs/.
- *Disclosure* is further categorized into informational or emotional disclosure.
- *Supportiveness* is further categorized into general, informational and emotional supportiveness.
Test set (Released!):
- UNLABELED TEST SET: About 3,000 unlabeled sentences, extracted from the top comment and posted to "POSTS".

Git repository: https://github.com/kj2013/claff-offmychest

Submission Link: EasyChair

Frequently Asked Questions (FAQs)

Q: Hello, where is the data?
- A: All of it is up now!
Q: Hello, how can I register?
- A: Post a tentative abstract to our EasyChair portal. See "Upcoming Deadlines" section below.
Q: But I haven't decided my approach yet?
- A: Submit an abstract anyway. We're trying to figure out how many participants to expect. You can always update your abstract and system paper (when you upload it).
Q: Oh but what are your metrics going to be?
- A: The standard stuff in binary classification tasks -- Precision, Recall, F1-Score.
Q: Do you really mean that everyone gets to present at AAAI 2020?
- A: Well yes! If you submit your system runs and a system paper then you can present your paper in the poster session and have your paper in our open access proceedings. There will still be peer feedback in which you can improve your camera ready paper. You do need to be registered for the workshop.
Q: Is there a kind of leader board to check the performance of leading models? Or could you please inform me a standard performance (e.g., baseline model’s performance a or human performance as a golden standard)?
- A: Unfortunately, we're not releasing a leader board before the workshop itself, so no one knows what the baseline performance is. But we're encouraging maximum participation and reminding you again that if you do write up a paper and submit your system results, you can present your poster at our AAAI workshop; if you're one of the best systems, you get a slot in the main session to present your approach. We'll also let you submit more than one run and up to ten runs if you want to experiment with different parameters and submit.
Q: Do you expect 2 separate papers for the two shared tasks?
- A: No, just one paper.
Q: For task 2, do we need to propose the solution or implement it as well?
- A: Task 2 is open-ended. If you do attempt it, we would expect an implementation, or at least a "walk-through".
Q: Are there travel grants to attend your workshop?
- A: Unfortunately we don't, but I know that AAAI does. It also allows you to volunteer so that you don't have to pay for registration. Do look into this. Hope to see your submission.
Q: I'm pretty sure I won't be able to attend your workshop. Can I still participate in the Shared Task?
- A: Make a submission, pay the workshop fee, get someone to put up your poster if you can. If we can, we'll try to let you Skype in.
Q: You are so awesome. Can we be friends in real life?
- A: Listen I'm just doing my job. But if you sign up for the workshop then we can be workshop buddies for sure!

Upcoming deadlines

December 13, 2019 - AAAI conference early registration deadline
~~December 13, 2019~~ December 18, 2019 - System runs due & System paper due
December 18, 2019 - Peer review process begins
January 17, 2020- Camera-ready contributions

February 7/8, 2020: All participants present at the Workshop at AAAI-20

Questions?

Contact Kokil Jaidka (jaidka at sas.upenn.edu) or Niyati Chhaya (nchhaya at adobe.com) and we promise to help.