Sonam Kharade - Research Projects

Systems Research

NASA RASPBERRY-SI:

RASPBERRY SI (Resource Adaptive Software Purpose-Built for Extraordinary Robotic Research Yields - Science Instruments) enable science instruments to autonomously adapt space lander and instrument software (and therefore its behaviors and actions) in response to newly discovered data on the planetary surface. The aim of this project is to increase the autonomy of a mission on the surface of another planet without the need for round-trip control data for human supervision. We also aim to increase the autonomy of the spacecraft in unknown and uncertain environments. This project will also increase the speed of scientific exploration via accurate task prioritization and also by reducing the number of interruptions in missions required by dynamically and carefully adapting to environmental and system changes during operation. We will demonstrate the effectiveness of our methods by deploying and optimizing state-of-the-art machine learning on the NASA testbed. This technology will enable learning-based autonomous planning and adaptation.

Performance evaluation, debugging, and optimization of highly configurable robotic systems

This project focuses on determining functional faults caused by misconfigurations in robotic systems through the lens of causality. We design and implement a framework termed CaRE (Causal Robotics Evaluation) to determine the root causes of the functional faults. Moreover after identification of root causes, parameter optimization algorithm will be developed to adapt the parameters in offline as well as online mode.

Formal verification of self adaptive systems through lens of causality

The formal verification faces dual curse of dimensionality in terms of search space in planning with increased time horizon and probabilistic model checking queries. In order to reduce the search space in planning we design a causality based algorithm to reduce the state space through causal inference of outcome causal model learned from experimental data. The proposed method is successfully demonstrated on the ocean waters landers testbed at NASA JPL.

Scalable Optimal Control

The scalable optimal control laws for stochastic systems in both continuous and discrete settings are developed in this work. The underlying principle is that in nonlinear Bellman equations, an exponential form of negative cost (referred to as the desirability function) results in linearity. Furthermore, given the desirability function, which is approximated using the sampling-based technique, the control input is determined analytically. The major objectives of this project are listed as

On Exact Embedding Framework for Optimal Actions in MDPs

This work deals with the embedding framework of Markov decision processes (MDPs) with discrete state and action space to find optimal actions. The optimal control problem of MDPs is efficiently tackled if it can be restructured into an equivalent linearly-solvable MDP (LMDP) through the process of embedding. In this work, we derive a constructive sufficient condition to arrive at an exact embedding solution; thereby, the embedded state cost matches the original system.

Optimal Control of Probabilistic Boolean Control Networks: A Scalable Infinite Horizon Approach

We define an augmented state space, and by exploiting Kullback-Leibler (KL) divergence the desirability function, we compute the optimal control using the path integral (PI) approach. We propose sampling-based techniques for the approximation of the PI and hence the optimal control of PBCNs. The sampling-based strategy is amicable to parallel implementation, thereby addressing the optimal control problem of large-scale PBCNs.

Optimal Control of Probabilistic Boolean Networks: An Information-Theoretic Approach:

In this work, we use an information-theoretic framework to compute optimal control of probabilistic Boolean network using path integral in finite time. In addition, an entropy-based sampling technique is developed to overcome the variance issue and use fewer samples.

Power system analysis and control

In recent years, the most researched complex uncertain system is the power systems where there exist many diverse uncertain parameters such as loads, electricity price, wind power generation, and photovoltaic power generation. Hence, the main theme of this work is to address various issues in the existing power system analysis based on the non-iterative method named holomorphic embedding power flow (HEPM). This exploits various concepts from complex analysis and provides a reliable physically existing solution to the power flow problem in various operating conditions.

The bifurcation point analysis in the generator outage scenario and in a more realistic view under non-uniform loading is performed using HELM. In addition, various iterative-based methods are proposed in the literature that lags in providing a reliable and accurate power flow solution for hybrid ac-dc systems. As a result, the next issue being addressed is the development of unified-HEPM to calculate the power flow solution of hybrid ac-dc systems.
Moreover, the power system has multiple generators, and loads connected together through network configuration should be synchronized where the generator synchronization is modeled as coupled oscillators in the Kuramoto framework. The conventional stability analysis iterative power flow solutions genuinely have convergence issues, that lag reliability, and scalability. To eliminate the dependency of the iterative-based load flow methods, a novel approach to adequately investigate the stability of the synchronized operating point in the multimachine power network using holomorphic embedding is developed.

Data-driven Approaches for System Analysis and Control

Effective Sequential Learning for Optimal Power Flow: A Soft Actor-Critic Approach

The traditional electric grid is evolving into a smart grid, with the integration of renewables, active loads, intelligent devices, etc. adding to the grid's complexity. Traditional tools used for grid operations and planning should be capable of addressing the evolving complexities. Optimal power flow (OPF) is one such tool underlying much of power system operations and planning. It determines the power generation required to meet an estimated demand while minimizing the generation costs and adhering to grid constraints. As a result, OPF forms a non-linear and non-convex constrained optimization problem. Traditional techniques like the interior point (IP) method optimize with a fixed set of constraints. Supervised learning techniques estimate the requisite generation solely based on input data and fail to incorporate the physical constraints of the device during prediction. In this work, we propose Soft Actor-Critic (SAC) based reinforcement learning to solve the cost-based OPF problem within grid constraints. The proposed method leverages the state-of-the-art agent making it effective, highly sample efficient, and not suffer from brittleness due to hyperparameters. Learning is accelerated by using historical data for initializing neural networks through behavioral cloning.

Coherency Identification in Multimachine Power Systems Using Dynamic Mode Decomposition

Synchronous generators continue to oscillate in multi-machine power systems in many coherent communities, and each community corresponds to the virtual generator. However, coherent communities can vary in response to different activities under different operating conditions. Hence it is necessary to conduct an online analysis however the resulting control actions are based on the system dynamical model. The system models formed using the first principle fail to represent the processes entirely throughout operating circumstances. Therefore a data-driven approach to system modeling is an effective resource for interpreting real-world activities and analyzing their effects in the future. For that, this work introduces a novel approach for data-driven modeling based on dynamic mode decomposition (DMD) for the coherency identification in a multimachine power system. In order to model the system more efficiently, both steady-state and transient data are considered. An augmented data matrix in the Hankel structure is passed as an input to the DMD algorithm for capturing the behavior of the system accurately. Incorporating the persistency of excitation and Hankel structure in DMD, a rank condition is derived to identify the coherency of generators in post-fault situations.

Voltage Collapse Boundary Detection in a Power System using Gaussian Process Regression

The consistently increasing demand, infrastructure restriction forces the power system to operate near its limits. The nonlinear power flow equations and network structure constrain the power transfer capability in large networks. The constraints violations result in voltage collapse blackouts in the worst case. In addition, the uncertainties arising due to the transmission line parameters and measurement noise complicate the power system analysis. Therefore, it is a great deal for a power engineer to explore voltage collapse boundaries under uncertainty to take preventive actions. In this work, a unified machine learning approach is developed to detect the voltage collapse boundary in the presence of uncertainty using the holomorphic embedding and Gaussian process regression. Holomorphic embedding provides the power flow solution space and voltage collapse points, while Gaussian process regression provides the analytical form of the boundary under uncertainty.