# Rohit Chowdhury

I am a Ph.D. candidate at the Department of Computational and Data Sciences, IISc, Bangalore. I was awarded the Prime Minister's Research Fellowship in August 2018, while joining IISc. I am associated with the QUEST (Quantifiation of Uncertainty in Engineering Science and Technology) Lab.

I am interested in the fields of autonomous navigation and data assimilation. My work focuses on developing algorithms and software that allow autonomous agents like Underwater Autonomous Vehicles (UAVs) and Unmanned Areal Vehicles (AUVs) to optimally navigate in stochastic and dynamic environments.

## Research Interests

- Autonomous Navigation
- Data Assimilation

- Markov Decision Processes
- Reinforcement Learning

## Education

**Ph.D. in Computational and Data Science (ongoing)**

**B.Tech in Mechanical Engineering**

2018- present

2012- 2016

# Projects

## Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows

We present a Reinforcement Learning (RL) based framework for computing a dynamically adaptable policy that minimizes expected travel time of autonomous vehicles between two points in stochastic dynamic flows. The data-driven dynamically orthogonal (DO) equations are utilized to forecast the stochastic dynamic environment. For planning, a novel physics-driven online Q-learning is developed. First, the distribution of exact time optimal paths predicted by stochastic DO Hamilton-Jacobi level set partial differential equations are utilized to initialize the action value function (Q-value). Next, the computed optimal policy is adaptively refined as the agent encounters flow during a mission. For the adaptation, a simple Bayesian estimate of the environment is performed and the inferred environment is used to update the Q-values in an ε−greedy exploration approach. We showcase the new RL algorithm and elucidate its computational advantages by planning paths in a stochastic quasi-geostrophic double gyre circulation.

This work has been accepted at the DDDAS 2020 conference.

## HPC-based Efficient Algorithm for Path-Planning

Navigation problems like planning optimal paths for an autonomous agent in stochastic environments can be addressed using Markov Decision Processes (MDPs). In case of dynamically changing environments (like a time-varying external velocity field, eg-wind/ocean currents), an additional time component in the state definition makes the state space much larger. This makes solving the MDP computationally expensive and time consuming.

At the same time, it is observed that such problems exhibit certain properties that make it suitable to exploit ideas of High Performance Computing (HPC) which make it possible to solve the given navigation problem at a fraction of the time that is currently reported in literature.

This project is currently in proress.

## POMDP Solution of the Tiger Problem with simulation

An implementation of Lark and White's pruning algorithm to solve the Tiger problem in POMPD literature. The corresponding video shows a simulation for a run where the time horizon is set to 5.

Link to source code on my github: https://github.com/rohit1607/POMDP_Tiger_Simulation

## Autonomous Navigation with Drone

Problem Statement: The autonomous drone must locate an entry in the building and enter it. Once inside, it must create a map of the environment.

The simulation (shown in video) was done using Gazebo and ROS. The drone has been equipped with a LIDAR and a Camera. The LIDAR is used to detect walls and obstacles. The camera is used identify an entrance flagged with blue markers, using a blob detection algorithm.

This was done as group project along with Chennam Revanth & Sourav Mishra

## A Study of the Ensemble Kalman Filter

A study of the Ensemble Kalman filter to estimate the states generated by the Lorenz 63 system of differential equations. Two cases- chaotic and periodic, were conisidered.

# Publications

**Chowdhury, R and Subramani, D.N.**

Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows* *

DDDAS 2020 Accepted.

# Contact

**Email: **rohitchowdhury@iisc.ac.in

**Address: **QUEST Lab, Dept. of Computational and Data Sciences, IISc, Bangalore