# Rohit Chowdhury

### Prime Minister's Research Fellow (Aug 2018)

### Dept. of Computational and Data Sciences,

### Indian Institute of Science, Bangalore

I am a Ph.D. candidate at the Department of Computational and Data Sciences, IISc, Bangalore. I was awarded the Prime Minister's Research Fellowship in August 2018 while joining IISc, Bangalore. I am associated with the QUEST (Quantifiation of Uncertainty in Engineering Science and Technology) Lab.

I completed my B.Tech. in Mechanical Engineering from NIT Warangal in 2016. I am interested in the fields of autonomous navigation and data assimilation. My current work focuses on developing algorithms and software that allow autonomous agents like Underwater Autonomous Vehicles (UAVs) and Unmanned Areal Vehicles (AUVs) to optimally navigate in stochastic and dynamic environments.

## Research Interests

Autonomous Navigation

Data Assimilation

## Education

**Ph.D. in Computational and Data Science (ongoing)**

**B.Tech in Mechanical Engineering**

Markov Decision Processes

Reinforcement Learning

2018- present

2012- 2016

# Research Projects

## Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows

We present a Reinforcement Learning (RL) based framework for computing a dynamically adaptable policy that minimizes expected travel time of autonomous vehicles between two points in stochastic dynamic flows. This framework consists of three key components-

The data-driven dynamically orthogonal (DO) equations to obtain a reduced-order stochastic velocity field in the form of realizations.

The Hamilton-Jacobi level set partial differential equations that enables us to compute an exact time-optimal path for a given realisation.

Reinforcement Learning- a framework that allows an agent to learn good decision sequences through experiences.

The data-driven dynamically orthogonal (DO) equations are utilized to forecast the stochastic dynamic environment. For planning, a novel physics-driven online Q-learning is developed. First, the distribution of exact time optimal paths predicted by stochastic DO Hamilton-Jacobi level set partial differential equations. These paths are treated as good experiences for a Q-learning agent and are utilized to initialize the action value function (Q-value) and estimate a good initial policy. Next, the computed policy is adaptively refined as the agent encounters flow during a mission. For the adaptation, a simple Bayesian estimate of the environment is performed and the inferred environment is used to update the Q-values in an ε−greedy exploration approach. We showcase the new algorithm and elucidate its computational advantages by planning paths in a stochastic quasi-geostrophic double gyre circulation.

The figures on the left show a distrubution of paths obtained by following two different policies in a stochastic, dynamic, double-gyre flow. In figure A, the policy was obtained using Dynamic Programming (used as a benchmark). In figure B, the policy was computed and updated using our proposed alogrithm.

This work has been published at the DDDAS 2020 conference. Please find the complete paper here.

## HPC-based Efficient Algorithm for Path-Planning

Navigation problems like planning optimal paths for an autonomous agent in stochastic environments can be addressed using Markov Decision Processes (MDPs). In case of dynamically changing environments (like a time-varying external velocity field, eg-wind/ocean currents), an additional time component in the state definition makes the state space much larger. This makes solving the MDP computationally expensive and time consuming.

At the same time, it is observed that such problems exhibit certain properties that make it suitable to exploit ideas of High Performance Computing (HPC) which make it possible to solve the given navigation problem at a fraction of the time that is currently reported in literature.

This project is currently in proress.

# Other Projects

## POMDP Solution of the Tiger Problem with simulation

An implementation of Lark and White's pruning algorithm to solve the Tiger problem in POMPD literature. The corresponding video shows a simulation for a run where the time horizon is set to 5.

Link to source code on my github: https://github.com/rohit1607/POMDP_Tiger_Simulation

## Autonomous Navigation with Drone

Problem Statement: The autonomous drone must locate an entry in the building and enter it. Once inside, it must create a map of the environment.

The simulation (shown in video) was done using Gazebo and ROS. The drone has been equipped with a LIDAR and a Camera. The LIDAR is used to detect walls and obstacles. The camera is used identify an entrance flagged with blue markers, using a blob detection algorithm.

This was done as group project along with Chennam Revanth & Sourav Mishra

## A Study of the Ensemble Kalman Filter

A study of the Ensemble Kalman filter to estimate the states generated by the Lorenz 63 system of differential equations. Two cases- chaotic and periodic, were conisidered.

# Publications

**Chowdhury R. , Subramani D.N. (2020)**

Physics-Driven Machine Learning for Time-Optimal Path Planning in Stochastic Dynamic Flows

In: Darema F., Blasch E., Ravela S., Aved A. (eds) Dynamic Data Driven Application Systems. DDDAS 2020. Lecture Notes in Computer Science, vol 12312. Springer, Cham. https://doi.org/10.1007/978-3-030-61725-7_34

# Courses

Numerical Linear Algebra

Numerical Methods

Introduction to Scalable Systems

Bioinformatics and Biology for Engineers

Data Assimiliation to Dynamical Systems

Parallel Programming

Numerical Solutions of Differential Equations

Autonomous Navigation

Numerical Optimization

Research Methods

# Teaching

# Advisor Information

**Assistant Professor**

**Dept. of Computational and Data Sciences**

**IISc Bangalore, India**

**Email: **deepakns.iisc.ac.in

# Contact

**Email: **rohitc1@iisc.ac.in

**Address: **QUEST Lab, Dept. of Computational and Data Sciences, IISc, Bangalore