Explaining Deep Reinforcement Learning Agents using Self-Attention Networks
Nishant Prabhu
Abstract
While the success of deep reinforcement learning (DRL) algorithms in several areas inspires confidence in its revolutionary nature, they often fall short on one important human aspect: explainability. An important measure of success of any intelligent system is the amount of trust a human would be willing put in its suggestions given that bad decisions could result in dire consequences. In many of its applications, the result of making a wrong decision could be loss of property or life. This calls for an imperative need to develop methods for explaining or interpreting DRL agents, so that the stakeholders may either be convinced of the agents’ behavior or override it when they are less confident. The very "black-box" nature of deep neural networks prevents several DRL agents from being inherently explainable. Consequently, a plethora of methods have been developed which help interpret one or more aspects of an agent’s behavior.
This work proposes a novel post-hoc explanation algorithm for visual DRL agents (ones that use pixel based inputs) operating in discrete action spaces. Given an agent that has already learnt a successful behavior policy in an environment, this work proposes a method to explain its decisions in any environment state regardless of the agent’s architecture or training methodology. Specifically, we train a multi-head self-attention based vision transformer to imitate the policy learnt by the trained agent by posing it as an action classification problem. After it has converged, we visualize the pixel-level attention probabilities learnt by the final layer of the self-attention network for each state encountered by following the trained agent’s policy in an episode. This makes our explanation method environment, agent architecture and training algorithm agnostic, allowing us to generalize to a wide variety of environments and agent architectures.
2. Improving Model-Based Reinforcement Learning for Robotics through Physics-Informed Neural Networks
Adithya Ramesh
Abstract
Reinforcement Learning (RL) agents can learn to solve sequential decision making problems through trial and error. In recent years, RL has been combined with powerful function approximators such as deep neural networks to successfully solve complex problems across a range of domains including robotics. However there remain some critical challenges that need to be overcome for us to see more successful deployments of RL based robotic systems in the real world. In this study we try to address one of these challenges, namely, sample efficiency, through the model-based RL approach. We learn a model of the environment and use it to generate imaginary trajectories, which are then used to update the policy and value functions. Through this approach, we are able to reduce the number of interactions needed with the actual environment. Intuitively, the quality of the learnt policy will depend on the quality of the imaginary trajectories, which in turn will depend on the quality of the learnt environment model. Thus, learning better environment models should lead to better policies. In this study we investigate if this is true.
In this work, we focus on the robotics domain, in particular, systems undergoing pure rigid-body motion. We assume full observability. We consider two different approaches to learning the environment model. The first approach simply uses standard deep neural networks. The second approach uses physics-informed neural networks that utilize the structure of the environment’s physics. We integrate each approach into a common model-based RL algorithm. We refer to the two variations as standard model-based RL and physics-informed model-based RL respectively. In addition, we also train a state-of-the-art model-free RL algorithm, Soft-Actor-Critic (SAC), to serve as a baseline.
We find that physics-informed model-based RL consistently outperforms standard model-based RL across environments. In simple environments, we find that the difference between the two approaches is small. In complex environments, the difference is significant. In simple environments, the performance of physics-informed model-based RL and SAC are comparable. In complex environments, physics-informed model-based RL significantly outperforms SAC. In the future, we plan to extend our work to robotic systems involving contact dynamics, dissipation and also partial observability.
3. LaVa: Latent Variable Models for Sample Efficient Multi-Agent Reinforcement Learning
Aravind Venugopal
Abstract
Multi-agent reinforcement learning (MARL) has widespread potential applications in real-world cooperative scenarios such as multi-robot coordination, smart grid optimization and autonomous driving, to name a few. Since each agent’s policy changes while learning, multi-agent environments are non-stationary with respect to each agent. Challenges arising from non-stationarity make learning difficult in environments where only a portion of the true state is visible to the agents (partially observable). High-dimensional inputs further complicate learning, as learning effective policies requires first learning good compact representations of the inputs. Thus, MARL tasks are associated with very high sample complexity. Despite recent advances in MARL, ensuring sample-efficient policy optimization through efficient representation learning remains a challenging question, rendering model-free MARL algorithms sample-inefficient. This limits their applicability in scenarios where it is costly to collect real-world data.
We propose Latent Variable Models for Multi-Agent Reinforcement Learning (LaVa), a novel, sample-efficient approach that utilizes an explicitly and efficiently learned model of environment dynamics to perform policy optimization using latent state representations. Efficient learning of dynamics is ensured using an exploration scheme that operates in the latent space to seek out expected future novelty of states. By separating representation learning from policy optimization, LaVa reduces environment interactions and accelerates learning. We perform empirical evaluation on complex, continuous control multi-agent tasks showing that our algorithm outperforms state-of-the-art model-free and model-based baselines in sample efficiency and final performance. Furthermore, our approach can be used with any multi-agent reinforcement learning algorithm.
4. Q-learning-based Actor-Critic Framework for Automatic Question Generation
Debargha Bhattacharjee
Abstract
Existing approaches in Automatic Question Generation (AQG) train sequence-to-sequence models in a super vised setup and generate questions from given passages and answers. However, such methods suffer from exposure bias and mismatch in the evaluation measures used for training and testing. Several works have used reinforcement learning-based techniques such as the Policy Gradient (PG) based REINFORCE algorithm to address some of these issues and fine-tune the model on a specific metric. However, these techniques work based on a global reward function where the model needs to wait till the generation of the entire sequence to update its parameters, limiting the model updates to only broader guidance.
In our work, we address these inherent issues in text generation problems like AQG by introducing a Q-learning-based actor-critic framework that uses fluency and semanticity-based rewards to aid sequence-to-sequence models in the task of question generation. However, the high dimensional discrete action space arising from the extensive vocabulary makes it challenging to leverage Q- learning-based methods for solving large-scale, real-world text generation problems. To address this issue, we use a two-step training procedure comprised of supervised pre-training of the actor followed by Q-learning-based joint training of the actor and critic. We also address the global reward problem by training a critic that can generate appropriate Q-values for question sub- sequences, making it possible to leverage such partially generated sequences to update the model parameters.
We present empirical results from experiments on the SQuAD dataset to establish our proposed framework's efficacy. We also analyze the questions generated using our proposed approach and compare them against the questions generated by the supervised learning based-baseline method to show the improvements provided by our proposed framework in predicting better quality questions concerning fluency and semanticity.
5. RL meets Droplet Microfluidics
Mohammad Shahab
Abstract
Droplet microfluidics is a platform for creating programmable and flexible devices that can execute a variety of processes on a single chip. Although fundamental functions such as droplet sorting, sequencing, and synchronization have been included in recent improvements in droplet-based devices, the design of microfluidic networks for specialized functionalities remains a research topic. This necessitates the creation of integrated devices capable of performing complicated operations and combining various functions on a single microfluidic chip. To find discrete microfluidic networks with particular functionality, a state-of-the-art Reinforcement-learning (RL) technique based on Temporal-difference Q-learning is employed in this study. The algorithm's implementation has been demonstrated using a combinatorial sorting problem, in which the goal is to sort all of the entering droplet sequences at the inlet for two separate droplet kinds. In a microfluidic network space, the program uses RL-based judgments to identify hidden network configurations. The superiority of the RL-based algorithm is established against random search based on execution time, and against Genetic Algorithms (GA) based on solution quality. This research might represent a first step toward the eventual objective of creating a unified droplet microfluidic design framework that can accommodate a large number of drops with different capabilities.
6. ExTra: Transfer-guided Exploration
Kartik Bharadwaj
Abstract
In this work we present a novel approach for transfer-guided exploration in reinforcement learning that is inspired by the human tendency to leverage experiences from similar encounters in the past while navigating a new task. Given an optimal policy in a related task-environment, we show that its bisimulation distance from the current task-environment gives a lower bound on the optimal advantage of state-action pairs in the current task-environment. Transfer-guided Exploration (ExTra) samples actions from a Softmax distribution over these lower bounds. In this way, actions with potentially higher optimum advantage are sampled more frequently. In our experiments on grid world environments, we demonstrate that given access to an optimal policy in a related task-environment, ExTra can outperform popular domain-specific exploration strategies viz. epsilon greedy, Model-Based Interval Estimation – Exploration Bonus (MBIE-EB), Pursuit and Boltzmann in rate of convergence. We further show that ExTra is robust to choices of source task and shows a graceful degradation of performance as the dissimilarity of the source task increases. We also demonstrate that ExTra, when used alongside traditional exploration algorithms, improves their rate of convergence. Thus, it is capable of complementing the efficacy of traditional exploration algorithms.
7. Causal Contextual Bandits with Targeted Interventions
Siddharth Nishtala
Abstract
We study a contextual bandit setting where the learning agent has the ability to perform interventions on targeted subsets of the population, apart from possessing qualitative causal side-information. This novel formalism captures intricacies in real-world scenarios such as software product experimentation where targeted experiments can be conducted. However, this fundamentally changes the set of options that the agent has, compared to standard contextual bandit settings, necessitating new techniques. This is also the first work that integrates causal side-information in a contextual bandit setting, where the agent aims to learn a policy that maps contexts to arms (as opposed to just identifying one best arm). We propose a new algorithm, which we show empirically performs better than baselines on experiments that use purely synthetic data and on real world-inspired experiments. We also prove a bound on regret that theoretically guards performance.
8. Reconstructing Unknown Water Networks Using Road Network Data
Anish Abhijit Diwan
Abstract
Water distribution networks serve a crucial role in the everyday sustenance of human settlements. Data regarding water networks hold immense value for various scientific, operational and developmental activities and are crucial to realize global goals of universal and equitable water access (as specified by the United Nations SDG 6). However, in reality, such data are often incomplete, outdated or unavailable due to a host of challenges arising out of technological, economic, and security limitations. In this work, we explore the idea of formulating network reconstruction as a sub-graph generation problem. We apply this methodology to the problem of predictive reconstruction of unknown water distribution networks by leveraging structural and functional correlations between a graph (road network) and its sub-graphs (water distribution networks). Correlations between road and water networks were studied to derive heuristic features for road network components. These features are then employed to postulate possible water distribution networks as subsets of the road network using methods such as spanning tree algorithms and graph neural networks.
9. Optimal Estimation of Parameters of a Water Distribution Network
Miheer Athalye
Abstract
A typical WDN (water distribution network) consists of pipes, junctions, pressure valves, reservoirs and tanks. There are a variety of models which effectively relate the attributes and layout of the network with its results in terms of flows, pressures and heads. This work is mainly concerned with developing techniques which can use heads and flow data from the WDN to estimate the parameters associated with modelling that network. In particular, we are concerned with accurately and efficiently estimating the major loss (pipe roughness) and minor loss coefficients of the pipes associated with modelling the network by the Hazen Williams formula. The formula relates the flow through a pipe with the difference in heads across the nodes at its ends by an equation using these coefficients. These parameters usually depend on age, material, network topology and other innate properties of the pipes and junctions of the network and once calculated can be used to solve the internal equations of the network associated with pressure, height, flow, and energy. The initial analysis is limited to networks which have a tree- type structure.
10. Controllability and External Stimulus in Brain Networks
Anand Uday Gokhale
Abstract
In the context of brain networks, there is a significant cost and difficulty associated with the collection of data in order to understand the system. As a result, the literature resorts to nonlinear oscillatory models for emulating brain networks. The effect of external stimulation on Brain Networks cannot be studied due to engineering constraints. In our work, we consider two nonlinear networked oscillators, namely the Hopf Model and the Kuramoto model. We validate the use of these models by showing that varying strengths of external stimuli lead to a trend in average controllability similar to that observed in Memory tasks in a real brain. Next, we use tools from information theory to quantify a functional connectivity graph and study the differences between the functional connectivity graph and the underlying adjacency matrix of the brain network. This work helps understand the effect of external stimulus on brain states and thus may be instrumental for the development of personalized stimulation protocols for medical treatment or performance enhancement.
11. Optimal Operation and Scheduling of Water Distribution Networks
Divyansh Raka
Abstract
The transportation of water for end-use is an energy-intensive operation. This work presents a web application developed for identifying the optimal schedule for operating a water distribution network. The application has been designed for water distribution networks consisting of pumps delivering water from the source to different overhead tanks. The overhead tanks considered for the application have separate inlet and outlet pipes, which is common in developing countries like India. The application reads from the user details of the network structure and consumer demands, solves a mathematical problem, and returns a schedule for operating the system that consumes minimum energy but can still satisfy the time-varying demands associated with the overhead tanks.
The underlying algorithm tries to provide a repeatable schedule such that water levels in the tanks at the beginning and end of the day are similar. The algorithm also tries to minimize the number of valve switches to reduce the manual work required in opening and closing the valves throughout the schedule. The original problem is non-linear because of the non-linear nature of the pump-operating curve and the system hydraulics. However, for the system considered, decoupling the network hydraulics from the mathematical optimization reduces the actual non- linear problem to a mixed-integer linear programming problem (MILP). The MILP is solved using the column generation approach with HiGHS and CBC solver. This approach substantially reduces the search space and thus increases the solution speed of the solver.
The interface of the application has been designed in such a way that utility providers can start using it with little or no training, and the maximum run time of the application to produce results has been kept under 3 mins to make it convenient for use in real-world scenarios. To use the application, a user can provide the network and demand details either as an excel file (following an excel template) or as inputs on a web application. The application has been tested on various rural water distribution networks. The resultant schedules in all cases were optimal, obtained in under 3 mins, and ensured no overhead tank had an overflow or ran dry throughout the schedule.
12. Data driven operation of water distribution network
Adhityan R
Abstract
Water distribution networks (WDN) in developing countries such as India are largely operated manually, resulting in inequitable supply and inefficient operation. Through data-driven scheduling of WDN, the near-optimal schedule for WDN is possible, which results in the equitable water supply to the demand points and enhances operational efficiencies. The Data required and the time to schedule large networks can be reduced with the help of the proposed heuristic approach.
13. Chemistry Inspired Molecular Representation for Property Prediction
Joe Bobby
Abstract
In recent years, there has been a steady rise in the novel emerging and re-emerging infections like Ebola, SARS, and H1N1. Pathogens are also becoming resistant to existing medications of common dis-eases due to their mutations. Hence, there is a need to develop novel strategies for drug discovery to find new and effective drugs. Hence, prediction of molecular properties using molecular structures is thus an important problem in drug discovery and drug repurposing. In this work, we propose a novel molecular representation approach using the concept of functional groups in chemistry to predict property of molecules using molecular structures. The approach is also compared with the state-of-the art deep learning model.
14. Genetic Drivers of Phenotypic Diversity
R. M. Rajeeva Lokshanan
Abstract
Amongst model organisms, Baker’s yeast, i.e., Saccharomyces cerevisiae, has been a favourite of many researchers in the field of genetics primarily due to its ease of handling and maintenance, the wide range of phenotypes that can be measured and the similarity it shares with the human genome. This has yielded many insights into how many biological processes function and are regulated. However, we still do not know about the roles of many of the genes and their mutations in the yeast genome. This is partly because of the sheer amount of diversity present amongst the different strains of the species. Here whole genome sequencing can help us immensely as it will highlight the differences between the various strains. In this work, we try to combine sequence data and phenotype data to infer the genes and their mutations that are responsible for a given phenotype.
15. A mathematical framework to integrate signaling networks with metabolic networks
Pavan Kumar S
Abstract
A holistic understanding of the biological interactions requires modelling integrated networks to elucidate normal and disease phenotypes. Despite signalling and metabolic networks being integrated, systems view of these networks allowed modellers to analyse them in isolation. In this work, a mathematical framework for integrating the signalling and metabolic networks is developed by integrating the constraint-based and ordinary differential equation (ODE)-based models. Particularly, metabolic networks are modelled using constrained-based modelling approach while signalling networks are modelled using a set of ODEs. At each time step, the information from the signalling network is fed into the metabolic network. The changes in the metabolic network are then looped back into the signalling network. This process is repeated at every time step. Further, the integration approach is corroborated by integrating insulin-glucagon signalling network with glucose metabolism. The developed framework captures the normal and disease phenotypes of the system. The algorithm developed is generic and can be employed to integrate signalling and metabolic networks for any biological system.
16. A network-based approach for studying metabolic interactions in a microbial community
Dinesh Kumar K B
Abstract
Bipartite networks can be used to represent metabolic reactions in an organism. These are called metabolic networks. Computational studies in microbial systems biology are often based on these metabolic networks of microbes. The existence of a variety of network-based algorithms makes it easier to employ metabolic networks to perform any analysis. The method we exploited in our research is based on a Python package, MetQuest, which employs graph-theoretical algorithms like breadth-first search to computationally simulate the metabolism of a microbe or a community given a set of seed metabolites to act as media, allowing us to determine potential metabolic interactions in a pairwise community and higher-order communities. Using this method, we studied two different microbial communities. 1) Ocular microbiome: Comparative study of patients infected with bacterial keratitis versus healthy eye. 2) Deep-sea hydrothermal vent microbiome from Guaymas basin. This analysis helps predict key microbes in a community, unique contributors of metabolic capabilities in the community, and mapping the effect of change in environmental metabolome on microbial interactions. The potential of this approach is not confined to these datasets alone but can be applied to any microbiome data to obtain similar insights. The observations can help researchers to focus wet- lab experiments on a specific aspect by pointing out possible interesting things to look at in a microbiome.
17. Calibration-Free Approaches for Modelling Reaction Systems using Spectral Data from Micro-reactors
Manokaran Veeramani
Abstract
Traditional methods for modelling reaction systems involve generating kinetic data, i.e., measurement of the concentration of different species involved in the reaction as a function of time in a batch reactor. However, even for homogeneous reaction systems, transport processes often confound data from these kinetic studies due to poor mixing characteristics in conventional batch reactors. Recently, continuous flow micro-reactors have been increasingly used for kinetic modelling studies because of their superior and enhanced transport properties. Also, monitoring reaction systems using spectroscopic techniques are quite common as they provide non-destructive and rapid analysis of the reaction mixture. Thus, micro-reactors integrated with spectroscopic instruments enable real-time monitoring of reactions and help in providing intrinsic kinetic information through minimal consumption of reagents.
The conventional approaches for modelling reaction systems from spectral data require labelled spectral data and corresponding concentration data to develop a calibration model, followed by estimation of kinetic parameters using computed concentrations. On the other hand, calibration-free methods use prior information about the system to identify kinetic parameters without the knowledge of pure component spectra. However, these methods require considerable knowledge of reaction systems (kinetic model, known pure component spectra) to guarantee a unique solution. This work proposes two calibration-free approaches for modelling reaction systems from unlabelled mixture spectral data based on prior information available about the system. These calibration-free approaches are corroborated using experimental data from Corning Advanced flow-reactorsT
18. Multivariate data analysis of chromium and its speciation in water using UV–Vis spectra
Divyang Deep Tiwari
Abstract
Chromium contamination in water has become a widespread problem due to its extensive uses in various industries. Chromium naturally occurs in valence states such as Cr(II), Cr(III) and Cr(VI). Chromium (III) is an essential micro-nutrient for the human body, while the Cr(VI) is highly toxic and carcinogenic. Determination of Chromium (Cr) speciation is very important due to its toxicity which depends on the oxidation number and its concentration. The aim of this work is to develop a simple, effective and rapid spectrophotometric method for the determination of metal ions concentration in aqueous solutions by using only its absorption spectra and this approach is tested on Cr(VI) system. 27 training samples of different concentration and pH of Cr(VI) has been prepared and their absorption spectrum is recorded using UV-visible spectrophotometer. Then molar absorptivity (also known as molar attenuation coefficient) of Cr(VI) species is estimated for the wavelength range of 200-800 nm using Beer-Lambert law. Then these molar absorptivity values are used with the absorption spectra in order to predict Cr(VI) concentration in 16 new test samples. This methodology is found to be effective with a mean absolute error of around 26 percent for these 16 test samples.
Abstract
The aim of bioprocess monitoring and control is to maintain an optimal extracellular environment for microbial growth and production. Offline concentration measurements cannot be used for online monitoring and control of bioprocesses. Near-infrared (NIR) spectroscopy is a widely used technique for online monitoring of bioprocesses. In this work, NIR spectroscopy is used for online control of bioprocesses in addition to online monitoring. In-situ NIR spectroscopy probe is used to measure spectra rapidly for performing feedback control of glucose concentration in a bioreactor. For achieving effective feedback control, a robust calibration model for predicting glucose concentration is developed using historical data from the previous (fed-) batches. A systematic way to update a calibration model using historical data is proposed and it is shown that the updated models improve the prediction of metabolite's concentrations during fermentation. A model-based feedback PID control is designed for controlling glucose concentrations in a bioreactor. The designed controller is used to control glucose concentrations using NIR spectra and the calibration model in an online manner. The proposed approach is demonstrated by controlling of the fermentation of Lactococcus lactis NZ9000 using the glucose feeding rate as a manipulated variable. The closed-loop feedback control maintains the glucose concentration in a bioreactor at 30 g/L for an initial 12 hours of fermentation. It is shown that the subsequent improvement of the calibration model helps better control of glucose concentration.
Abstract
An important characteristic of cancer is the heterogenous nature across tissues and subtypes, which requires personalized treatment or precision medicine. Genomic alterations have consecutive effects on the translation and transcription of proteins. An integrative multi-omics approach can provide a better perspective for predicting drugs for personalized treatment.
The goal of this study is to use omic information to determine a patient's therapeutic response and rank the drugs for personalized treatment. The models are trained to predict drug response values using interpretable machine learning algorithms, on cell line data, with an assumption that the cell lines imitate human cells in an artificial environment. We train machine learning models on mutation, copy number variation, and RNA data using multiple dimensionality reduction methods and ML algorithms.
Our tool uses different models for predicting the drug response of individual drugs to improve overall accuracy. We formulate a score to prioritize drugs for an individual using standard and personalized driver genes. The score considers biological information of driver genes and drug targets, to a curated ranking, this score can therefore be an absolute metric for each sample and a drug. The most suggested top-ranking drug for BRCA is MK-2206 (personalized) and OSI-027 (non-personalized). Overall, predictions of personalized drug response and prioritizing drugs for personalized treatment using our tool show similar results for breast cancer cell lines and TCGA-BRCA data.
21. Anomaly Detection
Jeswant Krishna K
Abstract
Timely anomaly detection is very crucial in many fields especially in the domain of aviation. Given a limited dataset consisting of multivariate time-series generated from various sensors and from multiple flights the task is to detect the anomalous flights in an unsupervised fashion. To tackle this task a complex systems approach has been investigated. The technique known as Multi-Fractal Detrended Fluctuation Analysis (MF-DFA) [[3], [2]] coupled with Generalised Procrustes Analysis (GPA) [1] has been used to build a metric. This metric gives a measure of dissimilarity between various datapoints and a simple thresholding on this metric can help in identifying anomalous flights. This complex systems approach to anomaly detection is also effective in accomplishing the objective which is evident from application of the methods to real-life flight data from Dassault Aviation Falcon 7X Bleed System. The results of this MF-DFA + GPA pipeline are compared against Autoregressive (AR) [4] modelling which leads to some interesting observations from the results observed from both techniques.
Abstract
Road crashes are causing nearly 1.5 lakhs of death every year. It is the fourth leading cause of death in India. One of the key barriers to reducing road crashes in India is the lack of comprehensive road crash data. However, all states in India record First Information Reports (FIRs) of road crashes. Tamil Nadu has both the databases – FIRs data and a comprehensive road crash data - Road Accident Data Management System (RADMS). First, we did text mining on FIRs data of the year 2017 and extracted 11 critical features from it - (1) Age, (2) Gender, (3) Profession, (4) Time, (5) Date, (6) Cause of crash, (7) Vehicle involved, (8) Collision type, (9) Road Facility involved, (10) Body parts injured, and (11) IPC sections. Then we compare the results of FIRs data with RADMS data. The study results suggest advantages and limitations of FIRs data and improvement in the RADMS data.
23. India Data Commons: A unified knowledge DB for all of Indian data
Senthamizhan V
Abstract
Publicly available Indian data from open sources, especially government-published data, are vital resources for students and researchers in a variety of disciplines. Unfortunately, processing these datasets is often tedious and cumbersome. Organizations follow distinctive practices for codifying datasets. Combining data from different sources requires mapping common entities (city, district, etc.) and resolving different types of keys/identifiers. This process is time consuming, laborious and done over and over. Our goal with Data Commons is to address this problem. India Data Commons synthesizes a knowledge graph from multiple data sources published by Indian government. It links references to the same entities (such as cities, districts, organizations, etc.) across different datasets to nodes on the graph, so that users can access data about a particular entity aggregated from different sources without data cleaning or joining. India Data Commons is equipped with a user-friendly and robust toolset in which the statistical variables can be visualized as line charts, scatter plots and choropleths. We also provide REST API, Google Sheet APIs and Colab notebooks to facilitate researchers in accessing our knowledge graph.
24. Robust preconditioners based on Hierarchical Matrices
Kandappan V A
Abstract
In this talk, we present a new preconditioner for dense linear systems arising from discretised integral equations in two dimensions. The developed preconditioner combines the traditional domain decomposition technique with hierarchical matrix representations, in particular the HODLR2D [1]. We apply this preconditioner to improve the conditioning of the system and thereby accelerate the convergence of the iterative solver. We present the preconditioner’s performance through numerical experiments on dense linear systems from discretised integral equations in two dimensions. We also compare the performance of the developed new preconditioner with a block diagonal preconditioner.
25. QUANTIFYING THE MAXIMUM POSSIBLE IMPROVEMENT IN 2^K EXPERIMENTS
Anusha Kumar
Abstract
This research formulates, and numerically quantifies the optimal response that can be discovered in a design space characterized by linear and two-way interaction effects. In an experimental design setup this can be conceptualized as the response of the best treatment combination of a 2^k full factorial design. Assuming prior distributions for the strength of main effects and interaction effects, this study enables practitioners to make estimates of the maximum possible improvement that is possible through design space exploration. For basic designs up to two factors, we construct the full distribution of the optimal treatment. Whereas, for more than two factors, we construct formulations for a lower bound.
26. Data driven monitoring of IoT enabled water distribution networks
Rohit Raphael
Abstract
Water distribution networks (WDN) throughout the country are poorly managed. There is very limited data being collected from these networks. While most other systems like communication and power grids are subject to drastic developments, water networks are still following traditional operation schedules and methods till date. Proper monitoring of water networks is essential not only for ensuring equitable and continuous water supply to end users but also to detect abnormalities in the system like leakage, pipe volume reduction from scaling, silt deposition and corrosion. Deposition of debris, silt and rocks reduces the usable volume of tanks and can cause crucial water shortage in large communities. This work devises a system comprising of level, flow and associated parameter monitoring systems where we check the conditions of water distribution networks. This work shows a cheap and reliable solution for WDN parameter measurement which is essential for sustainable utilization of water. By combining Internet of Things (IoT) and related free spectrum wireless transmission techniques, we can build a telemetry system which collect, analyze and visualize the data from the WDNs. Free spectrum technologies like LoRa and HC-12 are used for enabling remote monitoring of water tanks and pipelines. Here we emphasize non-intrusive sensing methods also thereby reducing the difficulties arising due to disruption in water supply during installation and maintenance of the system. Low cost monitoring and easily scalable design are added advantages of the proposed work.
27. Real time monitoring of Brix Value using Smartphone Based Refractometer.
Aditya Raman
Abstract
The work demonstrates a smartphone-based refractometer for analysing the brix value of a given sample. The device consists of an optical coupler which is designed on the principle of total internal reflection (TIR) of the incident light from the phone screen. The images generated from the TIR is recorded by the front facing camera of the smartphone, and is analysed using the model developed to detect the brix value of the given sample. For calibration water-sucrose solutions were used. The work tries to present a cheaper alternative to the handheld refractometers commercially available, and exploit the potential of smartphones for analytical purposes.
28. A Comparative Analysis of Topological and Conventional Methods for Time Series Segmentation
Shubham Kashyapi
Abstract
Segmentation involves partitioning a time series into a sequence of segments, where each segment corresponds to a distinct pattern of behavior. In most of the existing literature, this is achieved by fitting specific kinds of models (such as polynomial regression or AR models) to each segment. Recently, with the advent of Topological Data Analysis (TDA), many tools have been developed for extracting the underlying shape and structure of complex data. In particular, time series data can be embedded as point clouds to construct a network which can then be analyzed using tools from TDA such as persistent homology. This network encapsulates the multi-scale representation of the data in the form of a persistence diagram. It has been shown that persistence diagrams can capture a richer and more robust representation of the underlying dynamical system as compared to conventional techniques. In our work, we use this topological characterization to design a novel approach for time series segmentation, which identifies the regimes in the data. Using multivariate time series data from aircraft sensors, we empirically compare the results obtained from this new approach with those from conventional segmentation techniques.
29. Unsupervised Knowledge Hierarchy Building
Srijan Kumar Upadhyay
Abstract
Knowledge and understanding of a topic of interest are based on the analysis of observed data. The data finds its origin in sensor outputs or human observations and can be qualitative or quantitative. This observed data is analyzed at multiple levels of abstraction for patterns. The patterns thus detected at any level are termed primitives, which form the entities for analysis of the next level of abstraction. Thus, when performed repeatedly in this order, primitive detection and higher-order abstraction derivation lead to knowledge formation. In this project, a novel approach of fixed length primitive detection has been discussed for completely unsupervised learning. The compactness of representation forms the main criterion for this approach. The statistical method employed measures the change in the metric, compactness score, to be able to estimate the parameters and finally the primitives. This method has been applied to English language data in bit representation and was able to detect the alphabets in a completely unsupervised fashion. The project also touches upon variable length primitive detection.
30. Variable Selection using Unsupervised Learning
Abu Sai Kumar
Abstract
In recent years, Unsupervised feature selection methods have raised researchers’ interest in a variety of fields, owing to their ability to identify relevant features without requiring class label information. This research work focusses on spectroscopic data. Spectroscopic data is used for predicting concentrations of chemical samples. In general, spectroscopic data has many features (wavelengths) which are not relevant, so feature selection becomes important to remove redundant features. A method has been proposed to select important wavelengths, it uses singular value decomposition (SVD) to capture the local information of different species involved in the system. To validate the performance of the method, many spectra datasets (regression tasks) available online are used, PLS (partial least squares) models for multivariate calibration are built and evaluation metrics are compared for all wavelengths’ data and selected wavelengths data. The proposed method is also applied on other prediction tasks like classification on both spectra data and other data.
31. Self-Supervised Approach Towards Element Identification for Construction Progress Monitoring
Varun Kumar Reja
Abstract
Construction progress monitoring is one of the key tasks to be performed for a timely completion of a project. Conventional techniques of progress monitoring are error prone, requires human effort and are time consuming. Therefore, automating the process of quantification of progress is the way forward. Previously, researchers have experimented automating this task using heuristics-based approaches, which require to manually apply geometrical constraints and therefore require ample amount of hard coding and domain knowledge [1]. Recently, learning based approaches are being explored which takes the input as the as-built data from the projects, captured in the form of 3D point clouds. In this regard, computer vision coupled with machine learning is one of the primary technologies being utilised for element identification from point clouds [2]. However, the existing supervised approaches requires a lot of effort for manually labelling the training data. Hence, an approach which require less or no labels for learning the new data is the need of the hour. Therefore, this research aims to develop a self-supervised learning-based pipeline for element identification that learns the features from the construction point cloud data and classify them into various object classes. The proposed method is a self-supervised approach to find various classes of objects within a building point cloud. In the proposed method, first the data is pre-processed which includes the steps of down sampling and noise removal. Then the objects within a building point cloud are identified using unsupervised segmentation. The segmented objects are later given as input to feature extraction module, which uses contrastive learning approach. The learnt features are clustered to find various class of objects and provided with pseudo labels. These pseudo labels are considered as labels for a SVM based classification into six object classes i.e., wall, beam, column, door, window, and slab. The model is trained and evaluated on the S3DIS [3] dataset with the classes relevant to our problem. The results are extracted using the standard matrices for precision, recall, F1-score and overall accuracy.
32. Biological Network Reconstruction from Literature data using NLP
Harigovind R
Abstract
There are a lot of papers and articles that gets published every year in biomedical field. Hence a lot of information is available from which useful data has to be extracted.In this work we create a pipeline that extracts information from literature data using pre-trained NLP models like BioBERT,BERN,RoBERTa.We evaluate the developed model by testing it on different datasets and comparing it with state-of-the-art results.
33. Context-specific reconstruction of metabolic networks using transformer based NLP models
Sai N
Abstract
Constrained-based, genome-scale models can illuminate the molecular basis and provide a mechanistic understanding of the underlying phenotype. Since not all reactions present in a genome-scale model are expressed in different conditions, numerous algorithms have been developed to build context-specific models. These algorithms require reaction importance/confidence scores to be provided as input. Previous analyses have focused on extracting the confidence scores from transcriptomics data of the respective context. In this work, confidence scores of the reactions are predicted using literature evidence. Enormous growth of bio-literature data makes manual metabolic- network reconstruction infeasible. An automated pipeline coupling Named En- tity Recognition (NER) with constrained-based model-extraction algorithms is developed to construct context specific models. The pipeline uses external transformer-based NLP systems to perform NER tagging. Finally, to demonstrate the performance of the developed system, a metabolic network of small intestine enterocytes is constructed.
34. Perovskite based electrocatalyst identification using Machine Learning and SciBERT word embeddings
Arun Muthukkumaran
Abstract
Materials design for functional applications carried out using conventional trial and error methodologies is expensive and time consuming. Machine learning techniques can be used to design materials suitable for functional applications and discover materials for specific applications from existing literature. This study demonstrates a machine learning approach where a combinatorial analysis was used to design perovskite structures and a novel methodology was devised to calculate descriptor values which were used for property prediction of the designed perovskites. Further, natural language processing (NLP) technique was used to obtain word embeddings of perovskite materials. The obtained embeddings were used to identify perovskites that could be used as electrocatalysts. The candidate materials predicted by the NLP algorithm have been investigated for their electrochemical properties and they could be tried as electrocatalysts for the reactions discussed in this study.
Lokesh Rajulapati
Abstract
Traditionally, first principles modelling (white-box models) has been used to model the processes/phenomena in science and engineering. But, over the past few decades, due to the availability of large amounts of data, data-driven models (black-box models) have gained importance with advances in machine learning techniques. Hybrid models (gray-box models) are a combination of first principles and machine learning models. These models leverage the strengths of both the modelling paradigms and are known to have better predictive power, extrapolation capabilities etc. The goal of the current work is to provide a framework to place various techniques in hybrid modelling that have been documented in the literature.
36. MPC on Neural Networks: The Difference of Convex way
Sampriti Chattopadhyay
Abstract
In recent years, there is increased interest in applying advanced control strategies like Model Predictive Control (MPC) using Deep Neural Networks. This is due to the fact that many Neural Networks can be shown to be universal function approximators. However, the use of Neural Network models in MPC has issues at the time of optimization since the objective function is highly non convex and non smooth. Most algorithms do not provide convergence guarantees in such settings. Further, these algorithms are highly sensitive to the choice of hyperparameters and hence these need to be customized for each use case. In our work, we use an optimization method called Difference of Convex Programming and show that the objective can be solved as a sequence of Linear Programming or Quadratic Programming problems when used with some Neural Networks. Further we discuss that the algorithm takes finite time for convergence and is not sensitive to choice of hyper parameters.
Sriram R
Abstract
Oracle plays an essential role in Active Learning by annotating the unlabeled data acquired from query selection. In several scenarios, humans perform an oracle’s job, which is then referred to as Human-in- the-loop learning (HIL). Most of the previously researched topics on active learning treat the oracle as an unbiased data generating agent [4]. However, In the case of a human oracle, the dataset generated presents numerous challenges based on the ability of how humans interpret data or decision making, as it is riddled with heuristics and biases [5]. Bias towards a decision could be introduced when a human shows a tendency to base his/her decision on a particular subset of attributes alone or to make decisions based on relative judgment [1]. For example, when a human is asked to approve/reject a credit card application, he/she may bias the decision based on only a particular set of attribute values (Occupation, Salary) or could make a relative judgment with another rejected application.
This study aims to reduce the biases that occurred by a human oracle by employing a query procedure that drops specific attribute values while querying, thereby reducing the extent of relative judgment. To mimic the functionality of an oracle, the heuristics defined by Gigerenzer et al.[2] and Katsikopoulos et al.[3] were computed to annotate the Instances. Two Drop-out Mechanisms, namely the Random Drop-out Mechanism that drops attribute values while querying randomly and the Correlation-based drop- out mechanism which aims to drop attribute values based on the correlation of attributes with the labels in the previously labeled instance were proposed. Drop-out Mechanisms along with existing Active Learning algorithms were used to build prediction models for various classification tasks. It was found that the Random drop- out mechanism increased the performance of Active Learning algorithms when an oracle is said to have used a single decision strategy and the no. of predictors in the classification task was low. On the other hand, the correlation-based drop-out mechanism showed an overall improvement in the performance metrics of Active Learning algorithms irrespective of the no. of predictors in the Classification task.
38. Skip Connections leads to cliques and Cavities
Narendhiran R
Abstract
Historically, neural networks have been inspired by the biological mechanisms in human brain. Recent biological study reveals that in the human brain, information processing happens through synaptic networks containing an abundance of cliques of neurons in the brain. The brain processes stimuli not in a simple linear fashion, but by forming increasingly complex network characterized by a large number of functional directed cliques and cavities. We observe that a similar network structure has been arrived at independently when specific skip connections are used in neural networks. Also, we provide an overview of how the modern State of the art architectures are based out of these networks. Finally, we make the conclusion of how to define the structurally most efficient architecture and discuss potential future prospects that recent researches focus on.
39. Errors in Differentials
Om Sai Krishna Madhav Lella
Abstract
Propagation of errors in data with differentiation is not a widely studied topic. Understanding this subject can help us to solve various real-world engineering problems which in general involve non-linear variables like differentials. In this work, errors in the partial differential variables of Burger's equation (du/dx = 0.1d^2u/dx^2 - udu/dx) are analyzed and compared for different SNR values of errors in u. Errors in u are normally distributed and variables x and t are error-free. Characteristics of errors in du/dx, d^2u/dx^2, and udu/dx remain unchanged irrespective of the SNR values. Also, the covariance between the errors remains the same. This covariance matrix can be key in identifying the model accurately.
40. Drug-Drug Interactions using Graph Neural Network
Kaushik
Abstract
Adverse Drug-Drug Interactions remain a leading cause of side-effect induced mortality in healthcare. Identifying these from a molecular perspective is essential. Although several approaches have been proposed, this technique aims to use a graph representation of each drug to extract molecular and bond information. An auto-encoder model extracts a reduced dimensional space for each drug, which uses both labeled and unlabelled data. From the encoded representation, a supervised graph neural network approach was applied to predict the interaction between them; this model can be generalized for multiple drugs as it is independent of both size of each drug and the number of drugs. This model shows promising results even with a small set f data points and generalizes well with different data sets.
Prashant Govindarajan
Abstract
The advancements in generative modelling and unsupervised deep learning have contributed to the field of denovo drug design to a great extent. The aim of this project is to construct a conditional graph variational autoencoder (VAE) that can generate novel and drug-like molecules that can bind to a given binding site. Learned representations for the binding sites were obtained from the latent spaces of a separately trained graph VAE. Both ligands and binding sites are represented as graphs, which includes the spatial information of each atom in the case of ligands and residue in the case of proteins. The graphs are processed through graph neuralnetworks prior to VAE to maximize expressivity and capture structural information. The novel approach also uses sequential models (e.g., RNN, LSTM) for iterative edge generation, so that the presence of an edge between a pair of nodes is dependent on the previously generated edges. The performance of the model, once trained, can be assessed through metrics such as RMSD, docking score, QED, logP, and synthetic accessibility (SA), percentage of valid and unique molecules, and other appropriate scores, in addition to comparison with existing state-of-the-art networks. This project also focuses on demonstrating the smoothness of the latent representation of the trained model through latent perturbation. The progress of the project, future directions, and the challenges faced will be presented in this poster.
42. Metric Learning for comparison of HMMs using Graph Neural Networks
Rajan Kumar Soni
Abstract
Hidden Markov models (HMMs) belong to the class of double embedded stochastic models which were originally leveraged for speech recognition and synthesis. HMMs subsequently became a generic sequence model across multiple domains like NLP, bio-informatics and thermodynamics to name a few. Literature has several heuristic metrics to compare two HMMs by factoring in their structure and emission probability distributions in HMM nodes. owever, typical structure-based metrics overlook the similarity between HMMs having different structures yet similar behavior and typical behavior-based metrics rely on the representativeness of the reference sequence used for assessing the similarity in behavior. Further, little exploration has taken place in leveraging the recent advancements in deep graph neural networks for learning effective representations for HMMs. In this paper, we propose two novel deep neural network based approaches to learn embeddings for HMMs and evaluate the validity of the embeddings based on subsequent clustering and classification tasks. Our proposed approaches use a Graph Variational Autoencoder and diffpooling based Graph neural network (GNN) to learn embeddings for HMMs. The graph autoencoder infers latent low-dimensional flat embeddings for HMMs in a task-agnostic manner; whereas the diffpooling based graph neural network learns class-label aware embeddings by inferring and aggregating a hierarchical set of clusters and sub-clusters of graph nodes. Empirical results reveal that the HMM embeddings learnt through the Graph variational autoencoders and diffpooling based GNN outperform the popular heuristics as measured by the cluster quality metrics and the classification accuracy in downstream tasks.
43. Towards Responsible AI in Indian Scenario
Gokul S Krishnan
Abstract
The world is accelerating in terms of applying Artificial Intelligence (AI) on large-scale data across various sectors and therefore it is important to have policies and regulations in place for ensuring its responsible deployment. While the Indian organizations have put forth strategies for AI and Responsible AI, it is challenging to implement policies to ensure responsible deployment, monitoring and management of AI systems in Indian scenario. The main reasons for this are the social exclusion of citizens from digital technologies and the difference of semantics & standards of various data aspects in the Indian context, which vary ignificantly from the western context. This position paper attempts to shed light on these aspects and the importance of understanding the difference of standards and also puts forth some prospective measures that could be in place towards making Responsible AI possible in India.
44. Detecting Vehicles on the Edge: Knowledge Distillation to Improve Performance in Heterogeneous Road Traffic
Manoj Bharadhwaj
Abstract
The drastic growth in the number of vehicles in the last few decades has necessitated significantly better traffic management and planning. To manage the traffic efficiently, traffic volume is an essential parameter. Most methods solve the vehicle counting problem under the assumption of state-of-the-art computation power. With the recent growth in cost-effective Internet of Things (IoT) devices and edge computing, several machine learning models are being tailored for such devices. Solving the traffic count problem on these devices will enable us to create a real-time dashboard of network-wide live traffic analytics. This paper proposes a Detect-Track-Count (DTC) framework to count vehicles efficiently on edge devices. The proposed solution aims at improving the performance of tiny vehicle detection models using an ensemble knowledge distillation technique. Experimental results on multiple datasets show that the custom knowledge distillation setup helps generalize a tiny object detector better.
45. Hierarchical Off Diagonal Low Rank Matrices for problems in higher dimensions
Vaishnavi Gujjula
Abstract
Hierarchical matrices such as HODLR, HSS, H^2, etc. are used in constructing approximations and computing matrix operations for rank structured matrices. We introduce a new class of Hierarchical matrices for matrices arising out of the discretization of PDEs in higher dimensions, higher dimensional statistics, machine learning, etc. The matrix partitioning in a $d$-dimensional setting is done by constructing a $2^{d}$-tree over the underlying computational domain. We present the growth of ranks for various kinds of interactions. We identify the sub-blocks whose ranks do not scale with their size and low-rank approximate them. As a result, the computational complexity of construction and matrix-vector product scale almost linearly in the system size. We present various benchmarks and compare its performance with other Hierarchical matrices.