In work, we developed an evaluation platform built around the Overcooked game, designed to support Human-AI experiments. Overcooked is a two-player fully cooperative game. The system is shown as follows. 

Here, you're granted the ability to:

• Upload your weights

• Customize the human questionnaire

• Configure game settings

• And many more!

The code is available at Github Repo Link. In the repo, we also provide the pre-trained weights of SP, PBT, FCP, MEP and our COLE.

We visualize the trajectories of COLEs playing with middle-level partner (human proxy model) and expert partner (PBT). 

Note: All videos are sped up.

We visualize the trajectories of MEP and COLE playing with different humans at different starting positions. 

In the Coord. Ring layout, COLE slightly lags behind MEP but outperforms other baselines. Analysis shows MEP's success in human experiments is due to its predictable counterclockwise strategy, allowing easy adaptation by humans who simply choose the opposite direction. This predictability, however, may skew questionnaire responses about AI's contribution and teamwork. While MEP seems more understandable and contributive, it's largely due to its static strategy. Thus, MEP's performance drops with less intelligent partners, as illustrated in Fig.8.

Note: All videos are sped up.