Constructing Images to Elicit Specific Brain Activity

This piece, by Onno Berkan, was published on 03/14/25. The original text, by Luo et al., was submitted to NeurIPS 2023.

This CMU study introduces BrainDiVE (Brain Diffusion for Visual Exploration), which combines artificial intelligence with brain imaging to better understand how our visual system processes what we see. The main goal was to develop a data-driven approach to generating images predicted to activate specific brain regions without being limited by pre-selected categories or researcher bias.

A traditional way of studying visual processing in the brain relies heavily on carefully selected images chosen by researchers to test specific hypotheses. While this approach has helped identify brain regions that respond to categories like faces, places, and food, it may miss important aspects of how the brain processes natural scenes. BrainDiVE overcomes these limitations by using diffusion models (AI) combined with brain activity data from fMRI scans.

BrainDiVE connects two main components: a powerful AI model that can generate realistic images and a brain encoding model that predicts how the brain would respond to those images. When given a specific brain region to target, BrainDiVE can create new images that are predicted to activate that area strongly. This allows researchers to explore what kinds of visual features different parts of the brain respond to in a biased way.

Researchers showed that BrainDiVE could generate appropriate category-specific images for well-known brain regions that process faces, places, bodies, words, and food. They then demonstrated that the system could detect subtle differences between related brain regions, such as two different areas that both processes face but in slightly different ways. Perhaps most importantly, they discovered new functional subdivisions within known brain regions that hadn't been identified before, such as different activation patterns within food-processing areas.

Finally, the researchers showed the created images to human participants in an fMRI scanner. Activation patterns confirmed that BrainDiVE could successfully generate images with specific visual and semantic properties that matched what would be expected based on our current understanding of brain function.

This work is significant because it advances our understanding of how the human visual system processes complex information without being constrained by predetermined categories or artificial stimuli. While the researchers acknowledge that their method relies on pre-trained AI models and may reflect certain biases in the training data, they argue that BrainDiVE's data-driven approach could reveal new principles about how our brains organize visual information. This opens the door for a vast amount of neural encoding research.

Want to submit a piece? Or trying to write a piece and struggling? Check out the guides here!

Thank you for reading. Reminder: Byte Sized is open to everyone! Feel free to submit your piece. Please read the guides first though.

All submissions to berkan@usc.edu with the header “Byte Sized Submission” in Word Doc format please. Thank you!