NeuroAILab

Dimensionality of Intermediate Representations of Deep Neural Networks with Biological Constraints

It is generally believed that deep neural networks and animal brain solve a vision task by changing the dimen-sionality of input stimuli across deep hierarchy. However, so far the evidence shows that in object recognition task, deep neural networks first expand and then compress the dimensionality of stimuli, while primate brains do the opposite and first compress and then expand the dimensionality of stimuli across deep hierarchy. In this project, it is shown that if two biological constraints - namely the non-negativity of activities and energy efficiency in both activities and weights - are imposed on deep neural networks - the trend of dimensionality across deep neural networks and primate brain matches to each other and in both of these cases the dimension- ality of stimuli is first compressed and then expanded. This result shows that how neuroscience can help better understanding in artificial intelligence.

Code: https://github.com/aslansd/neuroinspired-vision 

Slide: https://drive.google.com/file/d/12LeMcfJXpwQBZGZT_LApU8G6Y-H-8ctE/view?usp=drive_link