Deep Reinforcement Learning-Assisted Convex Programming for AC Unit Commitment and Its Variants
Ahmed Rabee Sayed, Xian Zhang, Guibin Wang, Mostafa Shaaban, Mohammad Shahidehpour
IEEE Transactions on Power Systems, under review.
Embedded Files
Abstract
Unit commitment (UC) is one of the core decisions in electricity markets and system planning. Existing optimization methods either use DC approximation of network constraints or decompose and relax them into subproblems, which may lead to suboptimal/infeasible decisions or require execution times longer than those for the industry. Recent advances in machine learning techniques have proven their ability to find fast near-optimal UC decisions. However, they suffer from the curses of dimensionality and/or lack feasibility guarantees. This paper proposes a new hybrid deep reinforcement learning algorithm for the UC problem with AC network constraints (AC-UC). The problem is formulated as a modified-action Markov decision process model and the fundamental soft actor-critic (SAC) algorithm is enhanced to deal with hybrid action spaces. To ensure the solution feasibility, two action filters are designed based on logic constraints of generation units and sequential convex programming for the discrete and continuous actions, respectively. The hybrid SAC is improved by embedding the two filters and a prioritized experience buffer for fast convergence and stable learning. Simulation results on four test systems verify the effectiveness of the proposed approach and the superiority over traditional methods in computational efficiency and feasibility. Moreover, the presented method is extended to solve stochastic UC and security-constraint UC problems.
Keywords
Reinforcement learning, Convex programming, Unit commitment, Optimal power flow, Uncertainty
Page updated
Google Sites
Report abuse