Publications
*Two new AIED'24 accepted full papers coming soon*
Automated Long Answer Grading with RiceChem dataset (Shashank Sonkar, Kevin Ni, Richard G. Baraniuk) - Read on Arxiv
Marking: Visual Grading with Highlighting Errors and Annotating Missing Bits (Shashank Sonkar, Naiming Liu, Debshila B. Mallick, Richard G. Baraniuk) - Read on Arxiv
Does your LLM suck at math? Integrate the LLM with a programming tool and train it to use this tool for any math calculation. If the LLM thinks that its next response might require a calculation or a calculation verification, LLM will write the code for it, and then we execute that code and plug the result in for LLM to use it for a hallucination free response.
Paper: https://arxiv.org/abs/2309.12161
Code: https://github.com/luffycodes/Tutorbot-Spock-Phys
Model: https://huggingface.co/luffycodes/higgs-llama-vicuna-ep25-70b
Align your LLM to provide a scaffolded learning experience to students, rather than providing the answers straight away.
Follow up paper using RLAIF: Pedagogical Alignment of Large Language Models
Paper: https://arxiv.org/abs/2305.13272
Code: https://github.com/luffycodes/Tutorbot-Spock
MANER: Mask Augmented Named Entity Recognition for Extreme Low-Resource Languages (ACL SustaiNLP'23)
DUPE: Deduction under Perturbed Evidence: Probing Student Simulation/ Knowledge Tracing Capabilities of Large Language Models (AIED-LLM'23)
Knowledge graphs link entities through relations to provide a structured representation of real world facts. However, they are often incomplete, because they are based on only a small fraction of all plausible facts. The task of knowledge graph completion via link prediction aims to overcome this challenge by inferring missing facts represented as links between entities. Current approaches to link prediction leverage tensor factorization and/or deep learning. Factorization methods train and deploy rapidly thanks to their small number of parameters but have limited expressiveness due to their underlying linear methodology. Deep learning methods are more expressive but also computationally expensive and prone to overfitting due to their large number of trainable parameters. We propose Neural Powered Tucker Network (NePTuNe), a new hybrid link prediction model that couples the expressiveness of deep models with the speed and size of linear models. We demonstrate that NePTuNe provides state-of-the-art performance on the FB15K-237 dataset and near state-of-the-art performance on the WN18RR dataset.
Word embedding models learn semantically rich vector representations of words and are widely used to initialize natural processing language (NLP) models. The popular continuous bag-of-words (CBOW) model of word2vec learns a vector embedding by masking a given word in a sentence and then using the other words as a context to predict it. A limitation of CBOW is that it equally weights the context words when making a prediction, which is inefficient, since some words have higher predictive value than others. We tackle this inefficiency by introducing the Attention Word Embedding (AWE) model, which integrates the attention mechanism into the CBOW model. We also propose AWE-S, which incorporates subword information. We demonstrate that AWE and AWE-S outperform the state-of-the-art word embedding models both on a variety of word similarity datasets and when used for initialization of NLP models.
Arxiv link: https://arxiv.org/abs/2006.00988
PPT: Click on slides to view the presentation.
Knowledge tracing (KT) models, e.g., the deep knowledge tracing (DKT) model, track an individual learner's acquisition of skills over time by examining the learner's performance on questions related to those skills. A practical limitation in most existing KT models is that all questions nested under a particular skill are treated as equivalent observations of a learner's ability, which is an inaccurate assumption in real-world educational scenarios. To overcome this limitation we introduce qDKT, a variant of DKT that models every learner's success probability on individual questions over time. First, qDKT incorporates graph Laplacian regularization to smooth predictions under each skill, which is particularly useful when the number of questions in the dataset is big. Second, qDKT uses an initialization scheme inspired by the fastText algorithm, which has found success in a variety of language modeling tasks. Our experiments on several real-world datasets show that qDKT achieves state-of-art performance on predicting learner outcomes. Because of this, qDKT can serve as a simple, yet tough-to-beat, baseline for new question-centric KT models.
Arxiv Link: https://arxiv.org/abs/2005.12442
Poster: Click on poster to view the poster.
PCA visualization of question-assessment embeddings after training qDKT using fastText initialization. Points are colored blue for correct responses and red for incorrect responses.