Emin Orhan

CV | Research blog | GitHub | Google Scholar


I am currently a postdoc at the Center for Data Science at NYU, working with Brenden Lake. Most of my current work is driven by a modern version of the ancient nature vs. nurture question: I'd like to understand the limits of what we can learn from different types and amounts of data using generic but scalable learning algorithms and models. For example, what kind of visual representations can we learn from large-scale video data similar in kind and amount to the visual experiences of young children during early development? Is it possible to learn sophisticated internal models of the world, as manifested in intuitive physics judgments, for instance, from this kind of data using generic learning mechanisms? If not, can we maybe get there by increasing the data 10-fold, or 100-fold, compared to what a child would experience? If still not, what kind of inductive biases are necessary to learn such internal models? How does embodiment (being situated in a world and controlling a body to interact with it) change this picture? Recent progress in machine learning has opened up exciting new possibilities for addressing these fundamental questions at the intersection of machine learning and cognitive science. We can nowadays routinely train very large models on large, complex datasets or in realistic simulation environments, and probe the capabilities of the resulting systems in fine detail. This gives us a vastly improved capacity to grapple with the full scale and richness of the perceptual, cognitive, and motor problems faced by humans in the real world and to address really ambitious, old questions like the nature vs. nurture question. In my current work, I try to take advantage of these new exciting possibilities.


Orhan AE, Gupta VV, Lake BM (2020) Self-supervised learning through the eyes of a child. NeurIPS 2020 [Press 1] [Press 2] [3-minute summary] [1-hour talk]

Orhan AE, Pitkow X (2020) Improved memory in recurrent neural networks with sequential non-normal dynamics. ICLR 2020 [5-minute summary]

Orhan AE (2019) Robustness properties of Facebook's ResNeXt WSL models. arXiv:1907.07640

Orhan AE, Lake BM (2019) Improving the robustness of ImageNet classifiers using elements of human visual cognition. arXiv:1906.08416

Orhan AE, Ma WJ (2019) A diverse range of factors affect the nature of neural representations underlying short-term memory. Nature Neuroscience, 22, 275–283.

Orhan AE, Pitkow X (2018) Degeneracy, trainability, and generalization in deep neural networks. NeurIPS 2018 Workshop on Integration of Deep Learning Theories

Orhan AE (2018) A simple cache model for image recognition. NeurIPS 2018

Orhan AE, Pitkow X (2018) Skip connections eliminate singularities. ICLR 2018

Orhan AE, Ma WJ (2017) Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nature Communications, 8, 138.

Orhan AE, Ma WJ (2015) Neural population coding of multiple stimuli. Journal of Neuroscience, 35(9), 3825-41.

Orhan AE, Jacobs RA (2014) Are performance limitations in visual short-term memory tasks due to capacity limitations or model mismatch? arXiv:1407.0644

Orhan AE, Jacobs RA (2014) Toward ecologically realistic theories in visual short-term memory research. Attention, Perception, & Psychophysics, 76, 2158-70.

Orhan* AE, Sims* CR, Jacobs RA, Knill DC (2014) The adaptive nature of visual working memory. Current Directions in Psychological Science, 23(3), 164-70. (*equal contribution)

Orhan AE, Jacobs RA (2013) A probabilistic clustering theory of the organization of visual short-term memory. Psychological Review, 120(2), 297-328.

Orhan AE, Jacobs RA (2011) Probabilistic modeling of dependencies among visual short-term memory representations. NIPS 2011

Orhan AE, Michel MM, Jacobs RA (2010) Visual learning with reliable and unreliable features. Journal of Vision, 10(2):2, 1-15.

Work in progress

Please feel free to reach out to me to learn more about these ongoing projects:

  • Nature vs. nurture: How much of the early visual development in children can be explained in terms of generic learning mechanisms applied to the sensory data they receive? How much of it requires more substantive inductive biases? To address this modern variant of the age-old nature vs. nurture question, we apply self-supervised learning algorithms to a large-scale, longitudinal dataset of headcam videos from 6-32 mo infants and analyze the learned representations. Our first paper on this project was recently accepted to NeurIPS 2020. I am currently working on training self-supervised video models on the same dataset to understand how much intuitive physics can be learned in this generic way (without strong priors).

  • Embodied perception: How exactly do the perceptual representations emergent in embodied agents differ from those emergent in non-embodied agents? I try to tackle this question using modern simulation environments. Here is a recent research proposal I have written on this project.

  • Learning vs. memorizing: "Caching" is an effective method for improving the accuracy and robustness of a model, previously used in the context of language models (Grave et al., 2016; Khandelwal et al., 2019), image recognition models (Orhan, 2018), and in reinforcement learning (Blundell et al., 2016). Caching is conceptually similar to storing an example in episodic memory in animals (or "fast learning" in the complementary learning systems framework). In this project, I try to elucidate how caching works. One of the main goals of this project is to understand in what ways caching an example differs from learning that example in the parameters of a model. Here is a blog post I wrote a while back explaining part of the motivation behind this last question.

  • Data augmentation: Data augmentation is often crucial for the success of modern deep learning applications. In this project, I try to understand how exactly data augmentation improves the generalization capacity of deep learning models and why certain augmentations work better than others in this respect.


What I've been reading recently (non-fiction):

Dominion by Tom Holland

On Liberty by John Stuart Mill

A Treatise of Human Nature by David Hume

Not by Genes Alone by Peter Richerson and Robert Boyd

Every Life is on Fire by Jeremy England

Superforecasting by Philip Tetlock and Dan Gardner

Blueprint by Robert Plomin

The End of Doom by Ronald Bailey

Radical Markets by Eric Posner and Glen Weyl

Some podcasts I like:

Rationally Speaking

Many Minds

In Our Time

BBC Inside Science

Sean Carroll's Mindscape

Brain Inspired

The Curious Cases of Rutherford and Fry

The Rest is History


Astral Codex Ten


Bryan Caplan

The Shallowness of Google Translate by Douglas Hofstadter

Superintelligence: The Idea That Eats Smart People by Maciej Ceglowski

Never-ending Learning by Tom Mitchell et al.

The Bitter Lesson by Rich Sutton

More is Different by Philip Anderson