Paper Instructions
Each participating team, irrespective of its ranking, is encouraged to write and submit a system description paper. The following information pertains to the paper submission process:
Paper Submission Due: 19 February 2024
Paper Submission Requirements: https://semeval.github.io/paper-requirements.html
6 pages for a single-task system (not counting acknowledgments/references/appendices)
9 pages for multiple (sub)tasks (not counting acknowledgments/references/appendices)
Guidelines for SemEval System Papers: https://semeval.github.io/system-paper-template.html
Paper Format: https://acl-org.github.io/ACLPUB/formatting.html
Submission Site: https://softconf.com/naacl2024/SemEval2024/
Each system description paper's title should adhere to the following format: {TEAM NAME} at SemEval-2024 Task 7: {Paper Title}
For citing the task and the dataset, use the following BibTeX entries. Given that this edition is a continuation of previous editions, cite the task and dataset papers from the preceding edition:
Overview: https://drive.google.com/file/d/16QUul_US7sVxFxxc5DrtrUKXxZuQZBkv/view?usp=sharing
@inproceedings{numeval-report, title={SemEval-2024 Task 7: Numeral-Aware Language Understanding and Generation}, author={Chen, Chung-Chi and Huang, Jian-Tao and Huang, Hen-Hsen and Takamura, Hiroya and Chen, Hsin-Hsi}, booktitle={Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)}, year={2024}, publisher={Association for Computational Linguistics},}Task 1 Dataset Collection:
@inproceedings{chen-etal-2023-improving, title = "Improving Numeracy by Input Reframing and Quantitative Pre-Finetuning Task", author = "Chen, Chung-Chi and Takamura, Hiroya and Kobayashi, Ichiro and Miyao, Yusuke", editor = "Vlachos, Andreas and Augenstein, Isabelle", booktitle = "Findings of the Association for Computational Linguistics: EACL 2023", month = may, year = "2023", address = "Dubrovnik, Croatia", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.findings-eacl.4", doi = "10.18653/v1/2023.findings-eacl.4", pages = "69--77"}Task 1-1 Dataset:
@inproceedings{chen-etal-2019-numeracy, title = "Numeracy-600{K}: Learning Numeracy for Detecting Exaggerated Information in Market Comments", author = "Chen, Chung-Chi and Huang, Hen-Hsen and Takamura, Hiroya and Chen, Hsin-Hsi", editor = "Korhonen, Anna and Traum, David and M{\`a}rquez, Llu{\'\i}s", booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics", month = jul, year = "2019", address = "Florence, Italy", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/P19-1635", doi = "10.18653/v1/P19-1635", pages = "6307--6313"}Task 1-2 Dataset:
@inproceedings{ravichander-etal-2019-equate, title = "{EQUATE}: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference", author = "Ravichander, Abhilasha and Naik, Aakanksha and Rose, Carolyn and Hovy, Eduard", editor = "Bansal, Mohit and Villavicencio, Aline", booktitle = "Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)", month = nov, year = "2019", address = "Hong Kong, China", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/K19-1033", doi = "10.18653/v1/K19-1033", pages = "349--361"}Task 1-3 Dataset:
@inproceedings{mishra-etal-2022-numglue, title = "{N}um{GLUE}: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks", author = "Mishra, Swaroop and Mitra, Arindam and Varshney, Neeraj and Sachdeva, Bhavdeep and Clark, Peter and Baral, Chitta and Kalyan, Ashwin", editor = "Muresan, Smaranda and Nakov, Preslav and Villavicencio, Aline", booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = may, year = "2022", address = "Dublin, Ireland", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.acl-long.246", doi = "10.18653/v1/2022.acl-long.246", pages = "3505--3523"}Task 2 Dataset:
@inproceedings{10.1145/3459637.3482155,author = {Chen, Chung-Chi and Huang, Hen-Hsen and Chen, Hsin-Hsi},title = {NQuAD: 70,000+ Questions for Machine Comprehension of the Numerals in Text},year = {2021},isbn = {9781450384469},publisher = {Association for Computing Machinery},address = {New York, NY, USA},url = {https://doi.org/10.1145/3459637.3482155},doi = {10.1145/3459637.3482155},booktitle = {Proceedings of the 30th ACM International Conference on Information \& Knowledge Management},pages = {2925–2929},numpages = {5},keywords = {numeracy, machine reading comprehension, cloze test},location = {Virtual Event, Queensland, Australia},series = {CIKM '21}}Task 3 Dataset:
@article{huang2023numhg, title={NumHG: A Dataset for Number-Focused Headline Generation}, author={Huang, Jian-Tao and Chen, Chung-Chi and Huang, Hen-Hsen and Chen, Hsin-Hsi}, journal={arXiv preprint arXiv:2309.01455}, year={2023}}Explicitly articulate the underlying motivation for the design of your system. This includes detailing the specific problems your system addresses and the rationale behind its development.
Conduct a comprehensive analysis of your system's strengths and weaknesses. This should be presented in a structured format within the paper, offering valuable insights for future research.
Incorporate an error analysis section. This segment should highlight the scenarios where the system demonstrates optimal performance, as well as situations where it falls short. Additionally, examine any discernible patterns in the errors encountered.
For novices, examining system description papers that have received accolades, such as those winning the Best Paper Awards in SemEval (https://semeval.github.io/semeval2020-awards.html), is highly advantageous. Analyzing these papers can provide insights into the attributes of high-quality academic writing in this field.
Historical SemEval papers, which can serve as valuable references, are accessible at the specified location: https://aclanthology.org/venues/semeval/