Points

The submission of every team will be evaluated on each of the 5 objectives on the table above. The submissions will be ranked for each of the 5 objectives, and the points will be assigned to each team based on their rank (1st, 2nd or 3rd). Any team can collect the points from several categories. I.e. If a team's submission is 1st on objective 1 "Total", and 2nd on objective 5 "Carbon Emissions", the team will receive 440 + 110 pts.

There is a special category, "Non-RL", reserved for any teams that want to submit a control agent that is not based on Reinforcement Learning. The submissions in this category will be independent from the rest of the categories and will have its own ranking. Teams that submit a RL agent, are also allowed to submit a non-RL agent. It is possible to obtain points from both non-RL and RL categories (i.e. a team could be 1st on both a RL and a non-RL objective and receive points from both). If we do not receive any submissions for the non-RL category, we will add its points to each of the rows, 1 through 4, in the column "1st (RL)".

Scoring

When you run a simulation in CityLearn, it will return a set of metrics and scores once the simulation is over:

Metrics

  1. ramping: sum(|e(t)-e(t-1)|), where e is the net non-negative electricity consumption every time-step.

  2. 1-load_factor: the load factor is the average net electricity load divided by the maximum electricity load.

  3. average_daily_peak: average daily peak net demand.

  4. peak_demand: maximum peak electricity demand

  5. net_electricity_consumption: total amount of electricity consumed

  6. carbon_emissions: total amount of carbon emissions

Scores

  • Total - Average of the metrics 1, 2, 3, 4, 5, and 6 for the full simulated period (4 years)

  • Total Last Year - Average of the metrics 1, 2, 3, 4, 5, and 6 for the last year of the simulation

  • Coordination Score - Average of the metrics 1, 2, 3, and 4 for the full simulated period (4 years)

  • Coordination Score Last Year - Average of the metrics 1, 2, 3, and 4 for the last year of the simulation.

  • Carbon Emissions - Metric 6

Winner

We will rank all the teams on a leader board based on the total number of points received. The team with the highest number of points wins.

Prizes

Each team will receive a cash prize following the conversion 1 point = 1 USD.