Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning
Ahmed Rabee Sayed, Xian Zhang, Guibin Wang, Jing Qiu, Cheng Wang
IEEE Transactions on Power Systems, under review.
Embedded Files
Abstract
Increasing interdependencies between power and gas systems and integrating large-scale intermittent renewable energy increases the complexity of energy management problems. This paper proposes a model-free safe deep reinforcement learning (DRL) approach to find fast optimal energy flow (OEF), guaranteeing its feasibility in real-time operation with high computational efficiency. A constrained Markov decision process (CMDP) model is standardized for the OEF problem with a limited number of state and control actions and developing a robust integrated environment. Because state-of-the-art DRL algorithms lack safety guarantees, this paper develops a soft-constraint enforcement method to adaptively encourage the control policy in the safety direction with non-conservative control actions. The overall procedure, namely the constrained soft actor-critic (C-SAC) algorithm, is off-policy, entropy maximization-based, sample-efficient, and scalable with low parameter sensitivity. The proposed C-SAC algorithm validates its superiority over the existing learning-based safety methods and OEF solution methods by finding fast OEF decisions with near-zero degrees of constraint violations. The proposed approach indicates its practicability for real-time energy system operation and extensions for other potential applications.
Keywords
Safe learning, Optimal energy flow, Reinforcement learning, Fast control, Integrated energy systems, Soft actor-critic.
Test Systems for Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning.pdfPage updated
Google Sites
Report abuse