Join NumEval

1. Register (Now - Jan 30, 2024)

To participate in NumEval, kindly register using this Registration Form: https://forms.gle/LCEhpuRECyaggmsS6

Participants may opt to join one, two, or all of the tasks. We welcome submissions with results derived from NumEval's datasets.

For Task 1 and Task 2 participants, please submit a system description paper directly (Step 5).

For Task 3, we will have a private test set and an official evaluation. 

2. Explore NumEval Tasks (Now - Jan 20, 2024)

You will get the datasets for all tasks within a few days after your registration. Participants are encouraged to explore the training data to create their models. The use of external data or resources is not only permitted but strongly recommended. This exploration can continue until the evaluation phase begins.

3. Evaluation Period (Jan 10 - Jan 20, 2024)

We will release the test set for Task 3 on January 10th, 2024. Participants are expected to use their developed models to (1) formulate headlines and (2) delineate reasoning steps for the blanks provided in those headlines. Ensure your submission file mirrors the format of the training data. Details on the submission procedure will be sent to participants.

4. Human Evaluation (Jan 21 - Jan 31, 2024)

Participants will receive some submitted instances from other teams, and are expected to help us in evaluating from number accuracy, reasonableness, and readability aspects. More details will be released later.

5. Prepare System Description Paper (Feb 1 - Feb 19, 2024)

Teams are encouraged to draft a system description paper detailing their submission system, an analysis of their outcomes, and any notable insights. Please submit this by February 19th, 2024. Details regarding the paper submission process will be shared in due course.

• Paper Format: https://acl-org.github.io/ACLPUB/formatting.html

• Submission Site: https://softconf.com/naacl2024/SemEval2024/

6. Review and Feedback (Feb 19 - Mar 18, 2024)

Participants will review each other's papers and may offer guidance on papers that require further refinement in their writing.

7. Prepare Camera Ready Version (Mar 18 - Apr 1, 2024)

Following the review phase, each team must revise their paper in accordance with the feedback received and submit the final, camera-ready version.

8. Meet and Discuss at NumEval@SemEval-2024

Looking forward to seeing you at NumEval@SemEval-2024