Ahmed Rabee Sayed, Cheng Wang, Hussein Anis, Tianshu Bi
IEEE Transactions on Power Systems, 09 November 2022
Embedded Files
Abstract
Due to the increasing uncertainties of renewable energy and stochastic demands, quick-optimal control actions are necessary to retain the system stability and economic operation. Existing optimal power flow (OPF) solution methods need to be enhanced to guarantee the solution optimality and feasibility in real-time operation under such uncertainties. This paper proposes a convex-constrained soft actor-critic (CC-SAC) deep reinforcement learning (DRL) algorithm for the AC-OPF problem. First, this problem is standardized as a Markov decision process model to be solved by DRL algorithms. Second, the operational constraints are satisfied by a novel convex safety layer based on the penalty convex-concave procedure (P-CCP). Then, the control policy is updated by the state-of-the-art off-policy entropy maximization-based SAC algorithm. Therefore, the CC-SAC is a combination of data-driven and physics-driven approaches. The former speedups the solution time by predicting near-optimum control actions through a deep neural network. The latter effectively guarantees the solution's feasibility. Simulation results demonstrate the computational performance of the proposed CC-SAC to effectively find AC-OPF decisions with no constraint violation, zero optimality gap, and high speed up to 34 times compared to a state-of-the-art solver. The proposed approach indicates its practicability for power system real-time operation and marketing.
Keywords
Optimal power flow, Reinforcement learning, Soft actor-critic algorithm, Safe exploration, Difference-of-convex programming
Page updated
Google Sites
Report abuse