Emin Orhan

CV | Research blog | GitHub | Google Scholar

Research

I'm currently a research scientist at the Center for Data Science at NYU. My recent work has been motivated by a modern version of the nature vs. nurture question: I'd like to understand the limits of what we can learn from different types and amounts of data using generic but scalable learning algorithms. For example, what kind of visual representations can we learn from large-scale natural video data similar in kind and amount to the visual experiences of young children? Is it possible to learn sophisticated internal models of the world, for example as manifested in intuitive physics judgments, from this kind of data using generic learning algorithms? If not, can we get there by increasing the data 10-fold, or 100-fold, compared to what a child would experience? How does multi-modality or embodiment change this picture? Recent progress in machine learning has opened up exciting new possibilities for addressing these fundamental questions at the intersection of machine learning and cognitive science. We can now routinely train very large models on large, complex datasets and probe the capabilities of the resulting systems in fine detail. This vastly improves our capacity to grapple with the full scale and richness of the perceptual and cognitive problems faced by humans in the real world and to address truly ambitious questions like the nature vs. nurture question. In my current work, I try to take advantage of these exciting new possibilities.

Papers

Orhan AE (2022) Can deep learning match the efficiency of human visual long-term memory in storing object details? arXiv:2204.13061

Orhan AE (2021) How much human-like visual experience do current self-supervised learning algorithms need to achieve human-level object recognition?

Orhan AE (2021) Compositional generalization in semantic parsing with pretrained transformers. arXiv:2109.15101

Yoon K, Orhan AE, Kim J, Pitkow X (2021) Two-argument activation functions learn soft XOR operations like cortical neurons. arXiv:2110.06871

Orhan AE, Gupta VV, Lake BM (2020) Self-supervised learning through the eyes of a child. NeurIPS 2020 [Press 1] [Press 2] [3-minute summary] [1-hour talk]

Orhan AE, Pitkow X (2020) Improved memory in recurrent neural networks with sequential non-normal dynamics. ICLR 2020 [5-minute summary]

Orhan AE (2019) Robustness properties of Facebook's ResNeXt WSL models. arXiv:1907.07640

Orhan AE, Lake BM (2019) Improving the robustness of ImageNet classifiers using elements of human visual cognition. arXiv:1906.08416

Orhan AE, Ma WJ (2019) A diverse range of factors affect the nature of neural representations underlying short-term memory. Nature Neuroscience, 22, 275–283.

Orhan AE, Pitkow X (2018) Degeneracy, trainability, and generalization in deep neural networks. NeurIPS 2018 Workshop on Integration of Deep Learning Theories

Orhan AE (2018) A simple cache model for image recognition. NeurIPS 2018

Orhan AE, Pitkow X (2018) Skip connections eliminate singularities. ICLR 2018

Orhan AE, Ma WJ (2017) Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nature Communications, 8, 138.

Orhan AE, Ma WJ (2015) Neural population coding of multiple stimuli. Journal of Neuroscience, 35(9), 3825-41.

Orhan AE, Jacobs RA (2014) Are performance limitations in visual short-term memory tasks due to capacity limitations or model mismatch? arXiv:1407.0644

Orhan AE, Jacobs RA (2014) Toward ecologically realistic theories in visual short-term memory research. Attention, Perception, & Psychophysics, 76, 2158-70.

Orhan* AE, Sims* CR, Jacobs RA, Knill DC (2014) The adaptive nature of visual working memory. Current Directions in Psychological Science, 23(3), 164-70. (*equal contribution)

Orhan AE, Jacobs RA (2013) A probabilistic clustering theory of the organization of visual short-term memory. Psychological Review, 120(2), 297-328.

Orhan AE, Jacobs RA (2011) Probabilistic modeling of dependencies among visual short-term memory representations. NIPS 2011

Orhan AE, Michel MM, Jacobs RA (2010) Visual learning with reliable and unreliable features. Journal of Vision, 10(2):2, 1-15.

Miscellany

My personal blog

Clair de Lune by Guy de Maupassant

Do not go gentle into that good night by Dylan Thomas

A Poet's Advice to Students by e e cummings

The Bitter Lesson by Rich Sutton

Superintelligence: The Idea That Eats Smart People by Maciej Ceglowski

David Friedman

Bryan Caplan

Michael Huemer

FIRE