Volt-var control (VVC) is the problem of operating power distribution systems within healthy regimes by controlling actuators in power systems. Existing works have mostly adopted the conventional routine of representing the power systems (a graph with tree topology) as vectors to train deep reinforcement learning (RL) policies. We propose a framework that combines RL with graph neural networks and show that graph-based policies are significantly more robust to two typical data acquisition errors in power systems, namely sensor communication failure and measurement misalignment. Furthermore, we show that using a graph representation induced by power systems topology may not be the optimal choice and demonstrate that the choice of readout function architecture and graph augmentation can further improve training performance and robustness.

The problem of the efficient design of material microstructures exhibiting desired properties spans a variety of engineering and science applications. In this paper, we proposed a computational framework and show that physics-aware deep generative model can be used for the inverse design of material microstructures. Additionally, we demonstrate that deep neural networks, efficiently trained with multi-fidelity training data doubles as ideal surrogates for evaluating the physics-based constraints within such design frameworks.

The task of finding light dosage parameters for optimal printing conditions in two-photon lithography (an additive manufacturing technique) is often a routine and labor-intensive task. In this work, we applied a class of spatiotemporal deep learning model, specifically CNN-LSTM to automate the task of identifying optimal light dosage parameters. Incidentally, the same model can also be used for automated part quality monitoring during mass fabrication processes. The development of such models is important to help transform two-photon lithography from a niche laboratory technique into an industrialized technique.

In this work, we develop two attack models which perturbs the RL policy's actions with limited perturbation budget. We frame these attack models as constrained optimization problems and show that such attacks are feasible. The first attack model attacks the policy in a static manner using the gradient of the action probabilities. The second model plans for a sequence of attack by leveraging the dynamics of the policy. Our results demonstrate that by using the planning-based attack, the attacks end up having a much more severe effect than the static attack, under equivalent limited perturbation budgets. We further show how the analysis of these attacks are used to reveal potential weak points in the RL policy.

This work demonstrates how a sequential engineering design problem of designing a microfluidic device can be formulated and solved using conventional deep reinforcement learning algorithms. We show that to leverage conventional off-the-shelf RL algorithms, proper engineering of the environment is crucial, particularly the state and reward representations. Furthermore, popular techniques such as hindsight experience replay and transfer learning can also be employed to improve the data efficiency and generalizability of the RL design policy. Results from this study showed that combinatorically large design spaces, which typically arise in engineering design, can be intelligently explored and exploited using reinforcement learning approaches.