Olshausen - 2004 - Sparse coding of sensory inputs

Citation

Olshausen BA, Field DJ. Sparse coding of sensory inputs. Curr Opin Neurobiol. 2004 Aug;14(4):481-7. PUBMED

10 Word Summary

Sparse coding is a possible mechanism for efficient neural encoding.

Abstract

Several theoretical, computational, and experimental studies suggest that neurons encode sensory information using a small number of active neurons at any given point in time. This strategy, referred to as 'sparse coding', could possibly confer several advantages. First, it allows for increased storage capacity in associative memories; second, it makes the structure in natural signals explicit; third, it represents complex data in a way that is easier to read out at subsequent levels of processing; and fourth, it saves energy. Recent physiological recordings from sensory neurons have indicated that sparse coding could be a ubiquitous strategy employed in several different modalities across different organisms.

Notes

• Sparse coding is the method of using "simple bases" to represent complex signals
• Higher level neurons are less active than low level neurons
• Cortical areas "over-represent" sensory information
• The nervous system likely employs sparse coding because:
  • Patterns can be stored using local learning rules
  • It creates an explicit representation of the sensory input
  • It allows for sensitivity across many scales
  • It can be more energy efficient than other coding schemes
• A standard measure of sparseness is kurtosis:
• An argument counter to sparse coding is that neurons are trying to maximize entropy in order to minimize energy consumption.
• Experimental evidence for sparse coding exists in auditory and visual signals.