The participating team is required to submit one file in a specific format. We call it a "run file" or a "run" in short. Each team is allowed to submit 30 runs on the dev set, but up to 3 runs on the test set. Each run typically constitutes the results of a different system or a model.
The name of each submitted run file should follow the below naming format.
<TeamID_RunID.csv>
such that:
TeamID The teamID you used during task registering.
RunID can be an alphanumeric with a length between 2 and 9 characters.
For example, bigIR_bm25.txt.
The run file should have the following columns: [prompt_id, essay_id, holistic]. Each row indicates the holistic score of one essay. The expected run file is in csv format (comma separated).
For example:
2, 020210, 15This entry indicates that the system assigned a holistic score of "15" to the essay with essay_id "020210" from prompt_id "2".
The run file submission checker script is released on our Gitlab. It is mandatory to use this script to verify your run file (prior to submission).
The evaluation script is released on our Gitlab repository. It is mandatory to use this script to verify your model's performance.
Having done the registration steps mentioned here, you can submit your runs by following the steps below (these steps should be done by the team leader only as we will only approve the team leader in Codabench).
Create a Codabench account here.
Participate in the TAQEEM Task A competition on Codabench.
Once you join the competition on Codabench, the organizing team will review your registration and approve it within 24 hours. After approval, you can submit your runs.
Make sure that your run file follows the format described above. You should use the submission checker script to achieve that.
Convert your run file into a .zip file.
Login to Codabench.
Go to our "TAQEEM 2025 - Task A (Holistic Scoring)" competition page.
Go to "My Submissions" tab to and upload your .zip run file.
Leaderboard results of your run (in case no errors in the run file) will be shown in the "Results" tab. Only your team's best run will be shown. However, you can still see all the history of your submitted runs.
Number of runs: Please note that in the development phase, you can submit up to 30 runs. However, in the testing phase, you are allowed to submit 3 runs only. In both phases, the best run will be shown on the leaderboard.
Baseline: The run shown in the leaderboard under username salbatarni is the baseline.