I am a Ph.D. candidate at the Department of Computational and Data Sciences, IISc, Bangalore. I was awarded the Prime Minister's Research Fellowship in August 2018 while joining IISc, Bangalore. I am associated with the QUEST (Quantifiation of Uncertainty in Engineering Science and Technology) Lab.
I completed my B.Tech. in Mechanical Engineering from NIT Warangal in 2016. I am interested in the fields of autonomous navigation and data assimilation. My current work focuses on developing algorithms and software that allow autonomous agents like Underwater Autonomous Vehicles (UAVs) and Unmanned Areal Vehicles (AUVs) to optimally navigate in stochastic and dynamic environments.
Ph.D. in Computational and Data Science (ongoing)Indian Institute of Science, Bangalore
B.Tech in Mechanical EngineeringNational Institute of Technology, Warangal
Markov Decision Processes
Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows
We present a Reinforcement Learning (RL) based framework for computing a dynamically adaptable policy that minimizes expected travel time of autonomous vehicles between two points in stochastic dynamic flows. This framework consists of three key components-
The data-driven dynamically orthogonal (DO) equations to obtain a reduced-order stochastic velocity field in the form of realizations.
The Hamilton-Jacobi level set partial differential equations that enables us to compute an exact time-optimal path for a given realisation.
Reinforcement Learning- a framework that allows an agent to learn good decision sequences through experiences.
The data-driven dynamically orthogonal (DO) equations are utilized to forecast the stochastic dynamic environment. For planning, a novel physics-driven online Q-learning is developed. First, the distribution of exact time optimal paths predicted by stochastic DO Hamilton-Jacobi level set partial differential equations. These paths are treated as good experiences for a Q-learning agent and are utilized to initialize the action value function (Q-value) and estimate a good initial policy. Next, the computed policy is adaptively refined as the agent encounters flow during a mission. For the adaptation, a simple Bayesian estimate of the environment is performed and the inferred environment is used to update the Q-values in an ε−greedy exploration approach. We showcase the new algorithm and elucidate its computational advantages by planning paths in a stochastic quasi-geostrophic double gyre circulation.
The figures on the left show a distribution of paths obtained by following two different policies in a stochastic, dynamic, double-gyre flow. In figure A, the policy was obtained using Dynamic Programming (used as a benchmark). In figure B, the policy was computed and updated using our proposed algorithm.
This work has been published in the DDDAS 2020 conference proceedings. Please find the complete paper here: "Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows".
Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm
Autonomous marine vehicles play an essential role in many ocean science and engineering applications. Planning time and energy optimal paths for these vehicles to navigate in stochastic dynamic ocean environments is essential to reduce operational costs. In some missions, they must also harvest solar, wind, or wave energy (modeled as a stochastic scalar field) and move in optimal paths that minimize net energy consumption. Markov Decision Processes (MDPs) provide a natural framework for sequential decision making for robotic agents in such environments. However, building a realistic model and solving the modeled MDP becomes computationally expensive in large-scale real-time applications, warranting the need of parallel algorithms and efficient implementation. In the present work, we introduce an efficient end-to-end GPU-accelerated algorithm that (i) builds the MDP model (computing transition probabilities and expected one-step rewards); and (ii) solves the MDP to compute an optimal policy. We develop methodical and algorithmic solutions to overcome the limited global memory of GPUs by (i) using a dynamic reduced-order representation of the ocean flows, (ii) leveraging the sparse nature of the state transition probability matrix, (iii) introducing a neighbouring sub-grid concept and (iv) proving that it is sufficient to use only the stochastic scalar field's mean to compute the expected one-step rewards for missions involving energy harvesting from the environment; thereby saving memory and reducing the computational effort.
We demonstrate the algorithm on a simulated stochastic dynamic environment and highlight that it builds the MDP model and computes the optimal policy 600-1000x faster than conventional CPU implementations, making it suitable for real-time use.
The work has been published in the IEEE Journal of Oceanic Engineering and can be viewed here: "Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows Using a GPU-Accelerated Algorithm". A preprint of the same may be accessed here.
GPU-Accelerated Multi-Objective Optimal Planning in Stochastic Dynamic Environments
The importance of autonomous marine vehicles is increasing in a wide range of ocean science and engineering applications. Multi-objective optimization, where trade-offs between multiple conflicting objectives are achieved (such as minimizing expected mission time, energy consumption, and environmental energy harvesting), is crucial for planning optimal routes in stochastic dynamic ocean environments. The present paper extends our end-to-end GPU-accelerated single-objective Markov Decision Process path planner to compute optimal operating curves for multi-objective optimal planning. MDPs with scalarized rewards for multiple objectives are formulated and solved in idealized stochastic dynamic ocean environments with dynamic obstacles. Two simulated mission scenarios are completed to elucidate our approach and capabilities: (i) an agent moving from a start to target by minimizing travel time and net-energy consumption when harvesting solar energy in an uncertain flow; (ii) an agent attempting to cross a shipping channel with uncertain ship movement and flow. Optimal operating curves are computed in a fraction of the time that would be required for existing solvers and algorithms.
This work has been published in the MDPI Journal of Marine Science and Engineering and can be accessed here: "GPU-Accelerated Multi-Objective Optimal Planning in Stochastic Dynamic Environments"
POMDP Solution of the Tiger Problem with simulation
An implementation of Lark and White's pruning algorithm to solve the Tiger problem in POMPD literature. The corresponding video shows a simulation for a run where the time horizon is set to 5.
Link to source code on my github: https://github.com/rohit1607/POMDP_Tiger_Simulation
Autonomous Navigation with Drone
Problem Statement: The autonomous drone must locate an entry in the building and enter it. Once inside, it must create a map of the environment.
The simulation (shown in video) was done using Gazebo and ROS. The drone has been equipped with a LIDAR and a Camera. The LIDAR is used to detect walls and obstacles. The camera is used identify an entrance flagged with blue markers, using a blob detection algorithm.
This was done as group project along with Chennam Revanth & Sourav Mishra
A Study of the Ensemble Kalman Filter
A study of the Ensemble Kalman filter to estimate the states generated by the Lorenz 63 system of differential equations. Two cases- chaotic and periodic, were conisidered.
Game Theory- Ex-ante and Ex-post Fair Allocation
Implementation of the Probabilistic Serial and Recursive Probabilistic Serial Algorithms for fair resource allocation as described in "Best of both worlds: Ex-ante and ex-post fairness in resource allocation."
Link to source code on my github: https://github.com/rohit1607/GameTheoryMD_MiniProject
Chowdhury R. , Subramani D.N. (2020)
In: Darema F., Blasch E., Ravela S., Aved A. (eds) Dynamic Data Driven Application Systems. DDDAS 2020. Lecture Notes in Computer Science, vol 12312. Springer, Cham. https://doi.org/10.1007/978-3-030-61725-7_34
Chowdhury, Rohit, and Deepak Subramani.
"Optimal Path Planning of Autonomous Marine Vehicles in Stochastic Dynamic Ocean Flows using a GPU-Accelerated Algorithm."
arXiv preprint arXiv:2109.00857 (2021). (Under Review at the IEEE Journal of Oceanic Engineering )
Rohit Chowdhury, Atharva Navsalkar, and Deepak Subramani.
“GPU-Accelerated Multi-Objective Optimal Planning in Stochastic Dynamic Environments” (Under Review)
Teaching Assistant for Numerical Optimization Course, CDS, IISc Bangalore July - December 2021
Teaching Assistant for English at Kendra Vidyalaya Ongoing
Numerical Linear Algebra
Introduction to Scalable Systems
Bioinformatics and Biology for Engineers
Data Assimilation to Dynamical Systems
Game Theory and Mechanism Design
Numerical Solutions of Differential Equations