Submitting Your Entry to MCTF@AAMAS2024
All submissions must be made via the MCTF competition entry submission portal hosted on Codalab. Submissions made through any other platform or email will not be accepted.
Participants must be registered as mutually exclusive teams. Each participating team is allowed to submit only one final entry.
Submissions on the MCTF Codalab page can be made any number of times during the qualifying round. The last submission made by a participating team before the end of the qualifying round will be evaluated as the team's entry for qualification to the Finals round.
Submission Steps
Step 1: Register for a free Codalab account and go to the MCTF competition Codalab page here
Step 2: Click on the 'Participate' tab then select Submit/View Results to see the competition leaderboard on Codalab
Step 3: Format your submission following the steps given in the Formatting Your Submission section below
Step 4: Drop your submission zip folder in the 'Submit/View Results' under the Participate' tab on Codalab.
Testing Your Trained Policies Locally (on your machine)
Export the policies from your saved RLlib checkpoint using the competition_export_policy.py script. The script outputs three individual saved policy folders into a '/rl_test/policies/' directory. In the example below, we are exporting the trained policies that were saved in checkpoint_000006:
Usage: python competition_export_policy.py <path_to_saved_checkpoint>
Example: python competition_export_policy.py ./ray_test/checkpoint_000006/
The rl_test folder also contains a file called solution.py. This file lets you choose which trained policy or heuristic the agents in your team will use while playing the game during testing. More details are given in the section 'Updating Trained Policies in solution.py' below.
The testing script is inside the rl_test folder, called competition_test.py. You can run it with the following command:
Usage: python competition_test.py
Note: Your solution.py file should be in the same directory as the competition_test.py file
The testing script runs the game for a duration of 10 minutes of game time and outputs the score that your team received.
Updating Trained Policies in solution.py
The solution.py file in the rl_test directory provides the basic layout for loading the trained policies for your agent team for testing . First import your learned policies from the policies directory created from step 1 above. In the compute_action section you will map an agent_id to a policy or heuristic which will return an action for that agent to take.
Formatting and Uploading Your Submission
Select and zip the following files:
solution.py (Don't change this file's name)
policy checkpoint folder that was generated by competition_export_policy.py
DO NOT USE tar, gzip, or any other compressed folder format.
Submit the zip file via the Codalab page
Once uploaded, your submission will automatically be tested against Easy, Medium and Hidden difficulty levels of 3-player opponent teams and the scores will appear on the leaderboard of the Codalab page.
The competition code is currently set up to allow for training using RLlib. If you would like to use different reinforcement learning library, contact the competition organizers via email mctf2024@gmail.com or MCTF@AAMAS2024 Slack Channel.