Co-located with EMNLP 2022

EvoNLP Shared Task:

Temporal Meaning Shift


We present TempoWiC, a novel temporal meaning shift task. Given a pair of sentences (or, in this case, tweets) and a target word (e.g. delta), the task consists of deciding whether the meaning of the target word is the same or not. Basically, the framing is the same binary classification as the original WiC (Word-in-Context) task but adapted so the temporal aspect is taken into account (tweets in each pair were selected from different time periods).

For example, we can observe a meaning shift happening to the word folklore in the following instance, where its meaning represents a recent music album in the second example.

(1) There's a thunderstorm outside so clearly it's the perfect time to watch videos about folklore monsters. (August 2019)

(2) Cardigan on folklore is my favorite song. I wish @taylorswift13 would love me (August 2020)

The task also provides the time span for each tweet, and we encourage participating teams to use this information to improve their systems. The usage of this time span (date) is optional, and there's a single ranking for the competition. However, for workshop submissions we expect to make a clearer distinction between systems that make use of dates towards better performance.


Important Dates

  • 15 April 2022: Trial dataset released

  • 13 June 2022 (postponed): Training dataset released and CodaLab competition page open

  • 1 August 2022: Test data released and evaluation phase starts

  • 12 September 2022: Evaluation phase ends

  • 16 September 2022: Results released

  • 12 October 2022 [Extended!]: System description paper deadline (encouraged for most submissions)

  • 31 October 2022: Reviews released

  • 11 November, 2022: Camera-ready deadline

  • 7 December 2022: EvoNLP workshop


How to participate

This task is hosted on CodaLab: https://codalab.lisn.upsaclay.fr/competitions/5360


Contact

For any doubt, please contact Daniel Loureiro (boucanovaloureirod [AT] cardiff.ac.uk) or Jose Camacho-Collados (camachocolladosj [AT] cardiff.ac.uk).