Embedded Files
Recent Research
Recent Research
Neural network driven applications like ChatGPT are vulnerable to hallucinations where they confidently provide inaccurate information. Given that AI promises to herald the fourth industrial revolution, it is critical to understand and overcome these vulnerabilities. Doing so requires creating trustworthy neural networks that drive the AI systems. Defining trust, however, is not trivial. My research interests revolve around providing a human-centric approach to understanding trust in neural networks that allow AI to function in society. Doing so allows us to state the following: 1) All neural networks must provide contextual and relevant explanations to humans, 2) Neural networks must know when and what they don’t know, 3) Neural Networks must be amenable to being intervened upon by humans at decision-making stage. These three statements call for trustworthy neural networks to be explainable, equipped with uncertainty quantification, and be intervenable.
Neural network driven applications like ChatGPT are vulnerable to hallucinations where they confidently provide inaccurate information. Given that AI promises to herald the fourth industrial revolution, it is critical to understand and overcome these vulnerabilities. Doing so requires creating trustworthy neural networks that drive the AI systems. Defining trust, however, is not trivial. My research interests revolve around providing a human-centric approach to understanding trust in neural networks that allow AI to function in society. Doing so allows us to state the following: 1) All neural networks must provide contextual and relevant explanations to humans, 2) Neural networks must know when and what they don’t know, 3) Neural Networks must be amenable to being intervened upon by humans at decision-making stage. These three statements call for trustworthy neural networks to be explainable, equipped with uncertainty quantification, and be intervenable.
Visual explainability is a subset within neural network interpretability research, specifically geared towards explaining neural network decisions to users. The large deep learning algorithms are often seen as 'black-box' models where the decisions are not understood. The goal of an explainability algorithm is to investigate the reasons behind 'Why?' a particular decision is made. However such 'why?' questions are hard to answer uniformly. Instead, we advocate for a contrastive approach where explanations must answer 'Why P, rather than Q?' questions...
Neural networks reason inductively - after having learnt necessary and sufficient patterns while training, they search for similar patterns at inference. Having discovered these patterns, they make their decisions on given data. According to the network, the patterns are the 'cause' that lead to the 'effect', in this case the decision. However, when the train and test distributions don't match, networks fail to make the right decisions. Research in the distribution shift generally falls under robustness. We advocate for an abductive reasoning approach - create a hypothesis and tests its validity without considering the cause. We propose Introspective Learning...
The science of uncertainty quantification (UQ) deals with assigning probabilities regarding decisions made under some unknown states of the system. Generally, unknown states can occur due to: (1) Lack of data, (2) Underspecified or incorrectly chosen models or test data distributions, (3) Noisy ground truth labels, or (4) Interventions during inference. By definition, any measurement of a quantity requires 'fixing' (or holding constant or intervening) some other quantity for systematic study. If such 'fixings' are conducted across all possible sets of interventions, then no uncertainty exists. Very rarely are all interventions feasible (or possible)...
Trust is an esoteric quantity that cannot easily be measured. To garner the trust of humans, the underlying ML models must lend themselves to certain attributes that fall under the umbrella terminology of trustworthiness. However, these attributes are functions of the scientific communities that consider them. For instance, in the field of autonomous vehicles, the algorithmic trust in the vehicle’s perception module is different from the moral trust that is placed upon it, which is again different from the governmental policy trust that the vehicle abides by. Prediction trust, specifically evaluates trust that can be placed in a particular decision. Uncertainty and trust differ from each other...
PointPrompt is the first visual segmentation prompting dataset based on the Segment Anything Model (SAM). It is a comprehensive collection of human-generated prompts across 6000 images corresponding to 16 image categories and 4 data modalities (natural, seismic, medical and underwater). The prompting data was generated in an interactive manner, and at each step of the prompting process, the generated mask and associated score were shown to the annotator so they could adapt their strategy or move on to the next image. We have compared the segmentation scores obtained by our 48 human annotators against several existing automated prompting methods, showing that human prompting is consistently superior...
Past Research
Past Research
In the past, I have worked on image understanding applications related to active learning, image quality assessment, visual saliency detection, and seismic interpretation.
In the past, I have worked on image understanding applications related to active learning, image quality assessment, visual saliency detection, and seismic interpretation.
Page updated
Google Sites
Report abuse