CityLearn

CityLearn is an OpenAI Gym environment for the easy implementation of reinforcement learning agents in a multi-agent demand response setting to reshape the aggregated curve of electrical demand by controlling the storage of energy by diverse types of buildings. Its main objective is to facilitate and standardize the evaluation of RL agents such that it enables easy comparison of different algorithms.

CityLearn allows to control the storage of domestic hot water (DHW), chilled water, and electricity. CityLearn also includes energy models of air-to-water heat pumps, electric heaters, and the pre-computed energy loads of the buildings, which include space cooling, dehumidification, appliances, DHW, and solar generation.

Motivation

Periods of high demand for electricity raise electricity prices and the overall cost of power distribution networks. Flattening, smoothing, and reducing the curve of electrical demand help reduce operational and capital costs of electricity generation, transmission, and distribution. Demand response is the coordination of electricity consuming agents (i.e. buildings) in order to reshape the overall curve of electrical demand.

Reinforcement learning (RL) has gained popularity in the research community as a model-free and adaptive controller for the built-environment. RL has the potential to become an inexpensive plug-and-play controller that can be easily implemented in any building regardless of its model (unlike MPC), and coordinate multiple buildings for demand response and load shaping. Despite its potential, there are still open questions regarding its plug-and-play capabilities, performance, safety of operation, and learning speed. Yet, a lack of standardization on previous research has made it difficult to compare different RL algorithms with each other, as different publications aimed at solving different problems. It is also unclear how much effort was required to tune each RL agent for each specific problem, or how well an RL agent would perform in a different building or under different weather conditions.

In an attempt to tackle these problems, we have organized this challenge using CityLearn, an OpenAI Gym Environment for the implementation of RL agents for demand response and reduction of carbon emissions at the urban level. The environment allows the implementation of multi-agent decentralized RL controllers.