Embedded Files
I am a third-year PhD student at the Language Technologies Institute at Carnegie Mellon University where I am advised by Fernando Diaz. My research lies at the intersection of Natural Language Processing, Human-Centered AI, and Society. I work on evaluating the socio-cultural competence (aka alignment) and real-world impacts of language technologies.
During my PhD, I have interned at Semantic Scholar at the Allen Institute for AI, with Maria Antoniak and Tal August. Before my PhD, I was a pre-doctoral researcher at Google Research, India and Technology, AI, Society, and Culture Team, working with Partha Talukdar and Vinodkumar Prabhakaran on fairness in the Indian context. Before that, I spent a year at Microsoft Research, India working with Sunayana Sitaram and Monojit Choudhury on scalable and interpretable multilingual evaluation and human-in-loop evaluation. I graduated from BITS Pilani with a B.E. in Computer Science in 2021.
Research Summary:
I develop new evaluations that are principled, user-centric, and mixed-methods for evaluating cultural competence and impacts of language technologies.
In my PhD, I have worked on evaluating models' cultural competence, particularly in long-form and creative generation tasks [EECC, Research Borderlands]. My ongoing projects include developing evaluations for cultural homogenisation, stereotyping, and narrative inconsistencies in model-generated personalized stories for diverse demographic identities.
These days I have been thinking a lot about the following future research directions:
(a) situated evaluation, or understanding how professionals and artists use and interpret AI outputs in their workflows, how its (sociocultural) incompetence impacts this process, and how we can use implicit and explicit user feedback to detect and analyse these patterns;
(b) intrinsic evaluation, or how cultural competence can be tracked during pretraining.
I am broadly interested in making language technologies more inclusive to users from diverse cultures and understanding how language, technology, and society interact and shape each other.
News:
[July 2025] I will be IC2S2 in Norrkoping, Sweden 🇸🇪, from July 21-24, then at the pre-ACL workshop hosted by Pioneer Center for AI in Copenhagen, Denmark 🇩🇰 on July 25-26, and finally at ACL in Vienna, Austria 🇦🇹 from July 27-August 1, presenting Research Borderlands. If you are at any of these places and are interested in culture, fairness, evaluation, or policy, or ❗hiring summer interns in relevant topics❗ (or have recommendations for things to see, do, eat) let's find a time to chat :D
[May 2025] My internship work "Research Borderlands: Analysing Writing Across Research Cultures" was accepted as one of the 16 (!) plenary talks at the International Conference on Computational Social Science (IC2S2) 2025. AND It has been accepted to ACL 2025 (main). See you in Sweden and Austria 🎉
[Nov 2024] Ananaya's paper on evaluating robustness in hate-speech prediction in the presence of ethnicity markers was accepted to the Safe Generative AI Workshop at NeurIPS 2024
[Oct 2024] Thrilled that my first paper from PhD on evaluating cultural competence of LLM in text generation settings was accepted to EMNLP Findings! See you in Miami 🎉
[Aug 2023] I started my PhD at LTI, CMU.
[Oct 2022] Work on fairness in the Indian context from Google Research and on scalable and interpretable multilingual evaluation from my internship at MSR, were accepted to AACL 2022.
[Oct 2021] I started a github repo to curate advice related to grad school applications and research. Please contribute!
DEI Efforts:
DEI efforts and advocacy have always been an integral part of my life and career. My volunteer work profoundly shapes my views around access and the impact of technology and social opportunities.
I am an organiser at Queer in AI, where I help run our workshops and other initiatives to promote inclusion in the ACL community. Before that, I was co-organizing for WiNLP (Widening NLP), an organisation that supports underrepresented groups in NLP. I co-organize the mentorship program at LTI and frequently serve as a mentor for initiatives aimed at introducing people to research, both within and outside CMU. In my undergrad, I worked for educational and mental health initiatives for underprivileged kids for over three years. I am always looking for opportunities to do my bit to make the ACL, ML, and STeM communities more welcoming to everyone.
The interaction of society and technology is drastically altering how opportunities and marginalisation for underrepresented communities can be and are being, created. The landscape of AI and NLP for societal applications has a lot of uncharted territories. We need to understand who our technology affects and how to ensure that we do not harm the communities we seek to benefit. Technology can only genuinely benefit society when the people for whom it is being created are included in the process. So, we need to empower and listen to diverse voices in and outside of the research communities.
Contact
Reach me at: shaily@cmu.edu.
I am particularly happy to help undergraduate students, especially women, interested in NLP/ML, with exploring research, and applying for research internships or graduate studies (MS / PhD). I am open to talking about how I can help DEI efforts in the ACL and ML communities.
Page updated
Google Sites
Report abuse