CityLearn is an OpenAI Gym environment for the easy implementation of reinforcement learning agents in a multi-agent demand response setting to reshape the aggregated curve of electrical demand by controlling the storage of energy by diverse types of buildings. Its main objective is to facilitate and standardize the evaluation of RL agents such that it enables easy comparison of different algorithms.
CityLearn allows to control the storage of domestic hot water (DHW), and chilled water. CityLearn also includes energy models of air-to-water heat pumps, electric heaters, and the pre-computed energy loads of the buildings, which include space cooling, dehumidification, appliances, DHW, and solar generation.
Periods of high demand for electricity raise electricity prices and the overall cost of power distribution networks. Flattening, smoothing, and reducing the curve of electrical demand helps reduce operational and capital costs of electricity generation, transmission, and distribution. Demand response is the coordination of electricity consuming agents (i.e. buildings) in order to reshape the overall curve of electrical demand.
Reinforcement learning (RL) has gained popularity in the research community as a model-free and adaptive controller for the built-environment. RL has the potential to become an inexpensive plug-and-play controller that can be easily implemented in any building regardless of its model (unlike MPC), and coordinate multiple buildings for demand response and load shaping. Despite its potential, there are still open questions regarding its plug-and-play capabilities, performance, safety of operation, and learning speed. Yet, a lack of standardization on previous research has made it difficult to compare different RL algorithms with each other, as different publications aimed at solving different problems. It is also unclear how much effort was required to tune each RL agent for each specific problem, or how well an RL agent would perform in a different building or under different weather conditions.
In an attempt to tackle these problems, we have organized this challenge using CityLearn, an OpenAI Gym Environment for the implementation of RL agents for demand response at the urban level. The environment allows the implementation of single-agent (as a centralized agent) and multi-agent decentralized RL controllers.
Vázquez-Canteli, J.R., Kämpf, J., Henze, G., and Nagy, Z., "CityLearn v1.0: An OpenAI Gym Environment for Demand Response with Deep Reinforcement Learning", Proceedings of the 6th ACM International Conference, ACM New York p. 356-357, New York, 2019
Vázquez-Canteli, J.R., Ulyanin, S., Kämpf J., and Nagy, Z., “Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities”, Sustainable Cities and Society, 2018.
The MIT License (MIT) Copyright (c) 2019, José Ramón Vázquez-Canteli Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.