Projects

For image restoration using a geometrically stabilized Reverse Heat Equation, the blurring effect is modelled using the heat equation. Hence, the challenging problem of blind space-varying image deblurring can be solved by implementing the reverse heat equation to revert the blurring effects. Due to the unstable nature of the reverse heat equation, a stabilization effect is introduced by adding a normal component of the heat equation in the forward direction (Neumann boundary condition). A curvature-based stopping criterion appropriately stops the reverse heat equation without artefacts being introduced in the solution.

For image deblurring using Reverse Mean Shift, the image pixels are represented as points in a multidimensional space, forming intensity-based clusters. The key idea for deblurring is to increase the distance between different clusters (high inter-cluster separation), but at the same time, maintain close proximity within a cluster (low intra-cluster separation). This is achieved by an iterative application of the reverse mean shift filter along the cluster boundaries and the forward mean shift filter along the cluster interiors, thereby achieving denoising and deblurring.

Since the degree to which an image is enhanced by our algorithm is very subjective, we used the Peak Signal-to-Noise Ratio (PSNR) as a metric to judge the results from our algorithm. This metric mathematically expresses the ratio of maximum intensity in the image and the extent to which the image is distorted by noise (in our case blurring effect). We computed this metric using our deblurred image as the test image and the original sharp image in the dataset as the reference image.

Making scientific predictions based on historical time series data is known as time series forecasting. It involves development of algorithms and models for strategic predictions in order to drive future decisions optimally. In this project, we explored the scope of statistical models and neural networks in time series forecasting on datasets from a wide variety of fields, such as flux intensity of stars, stock prices and electric power consumption. We performed exploratory data analysis suited for time series data, consisting of stationarity tests, visualisation of moving averages, standard deviations and seasonal dependence, and plotting auto-correlation functions. Next we worked on predictions and forecasting on our datasets, by employing linear statistical models like auto-regression (AR), moving average (MA) and ARIMA. Finally, we graduated to using simple, stacked and bidirectional recurrent neural networks, LSTMs and GRUs for our prediction tasks. We also explored the performance of combining CNN and RNN layers together. All of these models perform differently, in terms of testing loss, based on the dataset being used.

We observed that across all datasets, the linear statistical models performed quite sub-optimally as compared to neural networks. For the power consumption data, the best performance was demonstrated by the stacked LSTM sequential model. We also observed that the stacked GRUs perform at par with the stacked LSTM model, with an order of magnitude faster convergence. The performance of simple and bidirectional GRU was nearly the same. For the stock prediction models, we observed that the stacked LSTM and stacked GRU models did not give good results, in spite of fine-tuning. The simple LSTM, bidirectional LSTM and CNN LSTM achieved similar losses on testing data. The best performance was given by the bidirectional GRUs. Best forecasting results were observed on the stars luminosity flux forecasting. This can be attributed to the presence of inherent periodicity in the data for variable stars. All the seven models performed at par for this dataset.

All the codes are implemented and made publically available at Github Repository time_series_forecasting. The detailed report of this project can be found here.

The problem at hand when determining the locomotion of a hexapod can be very well modelled as a Markov Decision Process (MDP). An MPD that characterizes a hexapod cannot have complete freedom since a hexapod can encounter actions that lead to collisions, falls, and other inaccessible locomotion for a real robot. Therefore, the actions of the hexapod robot should be constrained by several conditions such as acceleration, velocity, and torque constraints, which ensures the robot safe exploration. The adaptive locomotion control of a hexapod robot is a challenging task due to the high-dimensional observations, continuous state space and continuous action space. The Deep Deterministic Policy Gradient (DDPG) conquers this problem by employing Q-learning and Policy gradients. The details of this model can be found here.

Controlling agents with high dimensional sensory inputs is a long-standing challenge of Reinforcement Learning (RL). Deep Learning (DL) has actually made it possible to extract high-level features from raw sensory data by utilizing a wide range of neural network architectures (convolutional networks, multi-layer perceptrons, restricted Boltzmann machines and recurrent neural networks). Implementation of similar techniques in RL, although potentially very beneficial when dealing with sensory data, can be quite challenging from a DL perspective. A couple of these challenges are listed below:

• Typically, DL requires large amounts of hand-labelled data whereas RL requires us to learn from a scalar reward signal which can be sparse, noisy and delayed. The delay between actions and rewards can be quite long, as compared to the direct instantaneous association between inputs and targets in conventional DL applications.

• Another major concern is that the data in DL is almost always assumed to be independent, whereas, in RL, the states can be highly correlated. Moreover, this data distribution is subject to change as our RL algorithm learns, whereas DL assumes a fixed underlying data distribution.

The paper (Mnih et al., 2013) demonstrates how all of the above-listed challenges can be overcome using a convolutional neural network to learn successful control policies from sensory input data in complex reinforcement learning enviroments. This convolutional neural network is trained with a variant of the Q-learning algorithm and stochastic gradient descent is used to update the weights. An experience replay mechanism is implemented to take into account the correlations in the data. The methods are applied primarily to the Atari 2600 game, Pong. Results are also obtained on Atari games such as Breakout and Boxing.

We did a literature survey of the TD-Gammon Neural Network and Neural Fitted Q-Learning algorithms. Q-Learning algorithms suffer from substantial over-estimations of Q-values because they include a maximization step over estimated Q-values. These over-estimations, not only increase the number of episodes of training required to reach optimal policies, but can even lead to sub-optimal policies asymptotically. We propose using a Double Deep Q-Learning Network (DDQN) to tackle this problem. DDQN uses two separate Q-value estimators (2 neural networks), a target model for action selection and a primary model for action evaluation. Using these independent estimators, we can avoid the maximization bias by disentangling our updates from biased estimates. We used the Deep Q-Learning algorithm with Experience Replay, for learning to play Atari games at a level surpassing human performance. The agent receives raw gameplay data as pixels which are processed using Deep Convolutional Networks to extract relevant features and used to approximate the optimal action-value function. Learning off-policy through Deep Q-Learning coupled with Experience Replay which leverages recently experienced transitions for learning to help the agent learn in a fast and efficient manner.

High-quality object detection requires both precise localization (bounding box) and accurate classification of the target object. Object Detection is very relevant in the modern data-driven world, and has a multitude of applications, such as optical character recognition by traffic cameras to read number plates, tracking the motion of objects in real-time such as balls in various sports, and enabling self driving cars and automated robots. Modern object detectors can be broadly divided into two classes - single stage detectors

and two stage detectors. Single-stage methods perform a direct regression and classification by regular sampling grids on the whole image. On the other hand, two-stage detection approaches work by first generating a set of candidate proposals followed by classification and regression of these proposals. One of the most popular two-stage detectors is the Faster R-CNN. In this project, we were introduced to the dense local regression for precise target localization, which predicts multiple local box offsets, termed as dense box offsets, by a fully convolutional network. For classification, we introduced a discriminative RoI pooling that extracts features from various sub-regions of a proposal and performs adaptive weighting to obtain discriminative features. We reported results for instance segmentation, obtained by modifying the dense local regression branch of our two-stage detection method and utilizing instance mask annotations. Experiments were performed on two instance segmentation datasets: MS COCO and the recently introduced iSAID. Our method obtains consistent improvement over existing methods on both datasets. We also conducted a literature review of past-works in the field including R-CNN, Fast R-CNN, Faster R-CNN and the article D2Det: Towards High Quality Object Detection and Instance Segmentation covering Dense Local Regression, Discriminative RoI Pooling and Instance Segmentation. W reproduced the experimental results, along with introducing modifications to the deep network like adversarial

training, data augmentation, and cosine schedulers for learning rate schedule, to further improve the performance.

Quantum computation is the use of quantum-mechanical phenomena such as superposition and entanglement to perform computation. A quantum computer is used to perform such computation, which can be implemented theoretically or physically. Quantum computers could spur the development of new breakthroughs in science, medications to save lives, machine learning methods to diagnose illnesses sooner, materials to make more efficient devices and structures, financial strategies to live well in retirement, and algorithms to quickly direct resources such as ambulances. We experience the benefits of classical computing every day. However, there are challenges that today's systems will never be able to solve. For problems above a certain size and complexity, we don't have enough computational power on Earth to tackle them. To stand a chance at solving some of these problems, we need a new kind of computing. Universal quantum computers leverage the quantum mechanical phenomena of superposition and entanglement to create states that scale exponentially with the number of qubits, or quantum bits. To increase the computational power of a quantum computer, improvements are needed along two dimensions. One is qubit count; the more qubits you have, the more states can in principle be manipulated and stored. The other is low error rates, which are needed to manipulate qubit states accurately and perform sequential operations that provide answers, not noise. The end goal of this project was to give an introduction to quantum computation and quantum information theory from the perspective of theoretical computer science and getting a strong grip on linear algebra, discrete probability and theory of computation. It aims at getting a fair idea of the quantum circuit model of computation; basic quantum algorithms like Grover, Simon, etc; quantum entanglement. I did an extensive literature review of Quantum Computation: Quantum Circuit Model; Quantum Math basics; the Power of Entanglement; Grovers Algorithm; Quantum Query Model basics; Boolean Fourier analysis; Simons problem and Quantum Fourier Transform. My detailed report, Quantum Computation, has been published by the Maths and Physics Club of IIT Bombay on their official website.