2024 Competition:
For Human Authors

Important Dates

• 10. October, 2024: Deadline for submitting questions for the in-person mirror

• 11. November, 2024: Deadline for submitting questions for online mirror 

Prizes

• We will offer $50 for any submission set of 10 questions that is better than our staff-written questions.  For up to 10 submission sets.  If we get more than 10 such submission sets, the prize will be offered to the best ones ranked by the metric.

• A single author can win this incentive up to three times.

• In addition, we will also offer addition cash prizes for the five best questions ($50) if we have more than five distinct authors submitting qualifying packets. 

• The evaluation metric for written questions: 

  • https://drive.google.com/file/d/1byJ0_HYFBa-4y6SWHMf5JYC_cshE2JeG/view?usp=drive_link

Writing Requirements

The goal of the question writing competition is to write quiz bowl questions that machine learning models find difficult to answer, but skilled human players can still answer.

Submitted question requirements:

• Submitted questions should be tossups between 575 and 625 characters at 2-dot difficulty and adhere to quiz bowl style conventions.

• Submitted questions will be in the form of sets (10 tossups per set). You may submit up to 6 sets.  

• Submitted questions must not have previously been shared or submitted elsewhere. Please do not share any information on submitted questions with teams submitting models for the competition.

• See this doc for example submitted questions.

How to Submit Questions

• First, please register to participate in the question writing tournament: https://forms.gle/RoSzqJekTf3rAHhc7

Questions may be submitted either:

• Through this website: https://qantanew.web.app/editor. 

  • To make sure the questions are not too easy for the question-answering models, this interface checks whether and when models would able to answer the question.

• As a Google Doc formatted like this example packet submission and shared with the organizers: ying@umd.edu, yysung53@umd.edu, efleisig@berkeley.edu.

  • If you submit via a Google Doc, you're strongly encouraged to either use the interface to check the appropriate difficulty or to have some other mechanism to ensure that they're adversarial.

How to Write Questions

As we are particularly aiming for "adversarial types", make sure you adhere to the FAQ here

• To ensure the questions are high quality, our questions will be inspected by the set's editors. If your questions look good, they'll be used as is. If your questions are usable but require editing, they will be edited to fix any stylistic or factual errors.

Contact

If you have any questions, please contact us at qanta@googlegroups.com.