A few shot normalizing flows based method to learn embeddings by jointly training for differential privacy and utility.  The private embeddings can be released to a server model which can train further in a distributed setting. We achieve better privacy-utility tradeoff over DP-SGD while also reducing communication cost.

A pre-traning method to make language models structure-aware of the data on which they are trained. We used global tokens of sparse-attention transformers for header tokens. The result was improves accuracies in downstream information extraction tasks and interpretable attention patterns

Grievance redressal office receives many complaints from citizens which need to be sent to the correct ministry. The goal is to create a chatbot which can prompt the users for information while also classifying the complaint. We used the structural hierracrhies of the ministries to modify the retriver on Llama chat to work with a semi-supervised classifier using data programming.

Since many ICU and medical sub-routines are encoded in flowcharts, modern LLMs fail grasp the structure and hallucinate. We develop a belief based RAG and generate hyperbolic embeddings of nodes to model hierrarchical information