FURTHER (Red) vs LILI (Green)
FURTHER-MF (Red) vs LILI-MF (Blue)
Summary:
FURTHER is more effective than LILI by employing the limiting view via the average reward formulation.
FURTHER can scale to an environment with complex interactions (e.g., RoboSumo) and a large number of agents (e.g., Battle).