The Codenames AI Competiton

The Codenames AI Competition is based on and inspired by the board game Codenames created by Vlaada Chvatil.

The game offers a number of different facets that make it an exciting challenge and test-bed for natural language understanding systems: first and foremost, the lack of context means that a system must be capable of multi-modal language understanding. Nearly every clue found in Codenames can take on one of many different meanings -- e.g., COLD could refer to the temperature or the illness. Furthermore, the non-symmetric nature of the game means that two types of agents are required, those that need to be able to understand clues -- the guesser -- and the more challenging, selecting a clue from all possible English words such that a guesser can understand the intention.

Tracks

  • The Codemaster Track
  • The Guesser Track

Participants are allowed to submit to both tracks.

In the competition, bots from both tracks will be paired with all bots of the other kind, e.g., a Codemaster submission will be paired with all entered Guesser bots. The competition will be scored as:

  • If a bot loses (by choosing the Assassin or choosing all Blue tiles before choosing all Red tiles), it will get a score of 25
  • Otherwise, the score is the number of turns it takes to correctly find all Red tiles

Bots will be ranked by the average score of all of their pairings, ranked from lowest score to highest score.

Bots

Bots are Python 3 classes derived from classes found in the supplied competition framework: https://github.com/CodenamesAICompetition/Game -- all bots should come with a README with detailed description of any external resources required. Bots will be given a time limit of 30 seconds per turn.

Timeline

  • July 30th -- Competition submission deadline
  • August 26th -- Competition results presented at FDG 2019



The competition framework and supplied default bots can be found at:

https://github.com/CodenamesAICompetition/Game


Bots should be submitted via:

https://forms.gle/gNBGwqZG3yeAvH146


Thanks,

Adam Summerville, Andrew Kim, Maxim Ruzmaykin, Aaron Truong

Competition Organizers