How Good Are Deep Neural Networks at Modeling Vision?

This piece, by Onno Berkan, was published on 10/15/24. The original text, by Felix A. Wichmann & Robert Geirhos, was published by Annual Review of Vision Science in September, 2023.

This University of Tübingen study examines the relationship between deep neural networks (DNNs) and human vision, particularly in the context of object recognition. The researchers argue that while DNNs have shown impressive performance in computer vision tasks, they still differ significantly from human visual processing in several key ways.

The study highlights that DNNs lack robustness to changes in object pose and image distortions compared to humans. They also make errors that are inconsistent with human errors, suggesting fundamental differences in visual processing. Additionally, DNNs show less of a "shape bias" than humans, meaning they rely less on object shape for classification.

One obvious difference is DNNs' susceptibility to adversarial images - specially crafted images that can fool AI systems but are easily recognized by humans. This indicates that DNNs and humans use different features or decision processes for object recognition.

The researchers argue that while DNNs are powerful predictive tools, they currently fall short as explanatory models of human vision. DNNs' decision-making processes remain largely opaque, making it difficult to gain scientific insights from them. The authors suggest DNNs should be viewed as statistical models rather than mechanistic explanations of visual cognition.

The study emphasizes the importance of careful comparisons between AI and human vision. The authors believe vision science, with its strong scientific foundation, has much to offer the more engineering-driven field of deep learning. They argue that datasets and stimuli deserve as much attention as model architectures in advancing our understanding.

In conclusion, the researchers view DNNs as promising but not yet adequate models of human object recognition. They call for continued research to bridge the gap between artificial and biological visual systems.

Want to submit a piece? Or trying to write a piece and struggling? Check out the guides here!

Thank you for reading. Reminder: Byte Sized is open to everyone! Feel free to submit your piece. Please read the guides first though.

All submissions to berkan@usc.edu with the header “Byte Sized Submission” in Word Doc format please. Thank you!