Multi-Agent Reinforcement Learning with Emergent Roles

1. Ablation Study

Ablation study regarding L_I (the versatility and identifiability loss) and L_D (the specialization loss). Win rates at the end of training are shown, averaged over 8 random seeds.

Both LI and LD contribute to the significant outperformance, but the behavior specialization loss is more critical.

2. Game Replays

27m_vs_30m

27 Marines (left, controlled by ROMA) vs. 30 Marines.

6s4z_vs_10b30z

6 Stalkers & 4 Zealots (left, controlled by ROMA) vs. 10 Banelings & 30 Zerglings.

8m_vs_9m

8 Marines (left, controlled by ROMA) vs. 9 Marines.

MMM2

1 Medivac, 2 Marauders, and 7 Marines (left, controlled by ROMA) vs. 1 Medivac, 3 Marauders, and 8 Marines.

6z4b

6 Zerglings & 4 Banelings (lower left and upper right, controlled by ROMA) vs. 6 Zerglings & 4 Banelings.

10z5b_vs_2s3z

10 Zerglings & 5 Banelings (lower, controlled by ROMA) vs. 2 Stalkers & 3 Zealots.

2s3z

2 Stalkers & 3 Zealots (left, controlled by ROMA) vs. 2 Stalkers & 3 Zealots.

10m_vs_11m

10 Marines (left, controlled by ROMA) vs. 11 Marines.

5m_vs_6m

5 Marines (left, controlled by ROMA) vs. 6 Marines.

3. Emergence of Evolution of Roles

(a) Homogeneous Team

Role emergence and evolution on the map 10m_vs_11m (means of the role distributions at the first time step are shown, without using any dimensionality reduction techniques) during training. At this time step, ROMA learns to allocate roles according to agents' relative positions so that agents can form the offensive arc quickly using specialized policies.

(b) Heterogeneous Team

Role emergence and evolution on the map MMM2 (means of the role distributions at the first time step are shown, without using any dimensionality reduction techniques) during training. At this time step, ROMA learns to allocate roles according to agents' types.

Page updated

Google Sites

Report abuse