Research

• Intelligent Robot Learning Lab, University of Alberta (advisor: Matthew E. Taylor)

Which kind of human feedback is most useful under which conditions? I compare learning from demonstrations with preference-based RL as the human's task expertise, the feedback budget and the task conditions change, across several RL environments, and I test whether results obtained with synthetic (scripted) feedback hold with real people. In human studies, real human feedback produced a different kind of effect than synthetic feedback, not just a smaller one.

Paper: "Mind the Gap: Human and Synthetic Feedback Performance Diverge in Learning from Preferences and Demonstrations" - M. Ghayour, C. Muslimani, M. E. Taylor. Under review, AAAI-27 Student Abstract and Poster Program.

• Independent project, advised by Amir Behrad (Synthesis)

The same question for LLM reward models: reward models trained on human (HH-RLHF) and GPT-4-labelled (UltraFeedback) preferences agree far less with each other than with themselves, and pick different best-of-8 responses on 64% of prompts, against a 38-40% seed-to-seed floor.

• Intelligent Robot Learning Lab, University of Alberta

Vision-language-action models for human-robot collaboration on a Franka arm, in simulation and on the real robot; designing experiments for safe physical interaction with people.

• Graduate course project, University of Alberta

Does a person's decision time tell us how good their preference label is? A small human study on Walker (4 participants, 1,600 labels): labels were near chance, and weighting labels by decision time let a failed PEBBLE run learn.