Join NumEval

To participate in NumEval, kindly register using this Registration Form: https://forms.gle/LCEhpuRECyaggmsS6

Participants may opt to join one, two, or all of the tasks. We welcome submissions with results derived from NumEval's datasets.

For Task 1 and Task 2 participants, please submit a system description paper directly (Step 5).

For Task 3, we will have a private test set and an official evaluation. 

You will get the datasets for all tasks within a few days after your registration. Participants are encouraged to explore the training data to create their models. The use of external data or resources is not only permitted but strongly recommended. This exploration can continue until the evaluation phase begins.

We will release the test set for Task 3 on January 10th, 2024. Participants are expected to use their developed models to (1) formulate headlines and (2) delineate reasoning steps for the blanks provided in those headlines. Ensure your submission file mirrors the format of the training data. Details on the submission procedure will be sent to participants.

Participants will receive some submitted instances from other teams, and are expected to help us in evaluating from number accuracy, reasonableness, and readability aspects. More details will be released later.

Teams are encouraged to draft a system description paper detailing their submission system, an analysis of their outcomes, and any notable insights. Please submit this by February 19th, 2024. Details regarding the paper submission process will be shared in due course.

Participants will review each other's papers and may offer guidance on papers that require further refinement in their writing.

Following the review phase, each team must revise their paper in accordance with the feedback received and submit the final, camera-ready version.

Looking forward to seeing you at NumEval@SemEval-2024