Resources

This page presents a selection of papers, textbooks, videos, and other resources that have shaped how I think about and conduct my research. While the list is not exhaustive, it is intended as a useful starting point for those who wish to explore or deepen their understanding of cognitive science (particularly computational modeling) and machine learning (particularly reinforcement learning).

Computational modeling of brain and behavior

Tutorials and textbooks:

Ten simple rules for the computational modeling of behavioral data

Wilson and Collins in eLife (2019)A must-read for anyone who wants to start working on computational modeling of brain and behavior

Theoretical neuroscience: computational and mathematical modeling of neural systems

Dayan and Abbot 2001

Neuronal dynamics: from single neurons to networks and models of cognition

Gerstner et al. 2014

Computational foundations of cognitive neuroscience

Gershman 2025

Example papers:

Model-based influences on humans' choices and striatal prediction errors

Daw et al. in Neuron (2011)A classic example of a theory-guided experimental design to dissociate the predictions of different models

Human inferences about sequences: a minimal transition probability model

Meyniel et al. in PLOS Computational Biology (2016)A neat example of Bayesian modeling of neural and behavioral data

Reconciling novelty and complexity through a rational analysis of curiosity

Dubey and Griffiths in Psychological Review (2020)A great example of how normative modeling reconciles fragmented theories

Using large-scale experiments and machine learning to discover theories of human decision-making

Peterson et al. in Science (2021)An excellent example of using modern machine learning methods for modeling human behavior

Computational models of episodic-like memory in food-caching birds

Brea et al. in Nature Communications (2023)A successful example of integrating and testing traditional theories via computational models

Reinforcement learning and optimal control

Tutorials and textbooks:

Markov Decision Processes: Discrete Stochastic Dynamic Programming

Puterman 1994A math-heavy textbook to study the theoretical foundation of reinforcement learning

Reinforcement Learning: An Introduction

Sutton and Barto 2018The go-to reinforcement learning textbook

Algorithms for Reinforcement Learning

Szepesvari 2019A short textbook summarizing several key reinforcement learning algorithms along with their theoretical properties

Example papers:

Near-Bayesian exploration in polynomial time

Kolter and Ng in ICML (2009)A classic example of theoretically justified algorithm design for efficient exploration

Unifying count-based exploration and intrinsic motivation

Bellemare et al. in NeurIPS (2016)An elegant example of translating theoretical ideas into deep reinforcement learning

Learning to reinforcement learn

Wang et al. on arXiv (2017)The classic paper introducing meta reinforcement learning

Diversity is all you need: learning skills without a reward function

Eysenbach et al. in ICLR (2019)A great example of unsupervised, general-purpose skill-discovery

Discovering and achieving goals via world models

Menconda et al. in NeurIPS (2021)An excellent example of integrating several algorithmic modules for goal-conditioned reinforcement learning

My general go-to textbooks

Elements of Information Theory

Thomas and Cover 1991This is among the most well-written textbooks I have ever seen!

Information Theory, Inference and Learning Algorithms

MacKay 2003

All of Statistics: A Concise Course in Statistical Inference

Wasserman 2004It's pretty much exactly what its title claims!

The Elements of Statistical Learning

Friedman et al. 2009Deep learning and reinforcement learning aside, this can reasonably be regarded as a “complete” textbook on machine learning

Bayesian Reasoning and Machine Learning

Barber 2012This is particularly useful for probabilistic graphical models

Computer Age Statistical Inference: Algorithms, Evidence, and Data Science

Efron and Hastie 2016I LOVE this book! Particularly, the first few chapters on Frequentist, Bayesian, and Fisherian approaches to statistics

Some general resources to become a better researcher

To become better at presenting: "The Sense of Style" by Steven Pinker and "How to Speak" by Patrick Winston
To become better at doing research: "You and Your Research" by Richard Hamming and "Advice for Young Investigators" by Sam Gershman
To become better at understanding how science works: "The Structure of Scientific Revolutions" by Thomas Kuhn