Overview

In his paper on “the cognitive revolution,” George Miller dates the “moment of conception of cognitive science” to a 1956 symposium organized by the ‘Special Interest Group in Information Theory’ (Miller, 2003, pp. 142). This historical moment reflects the shared intellectual roots of information theory, which was founded by Claude Shannon (1948), and cognitive science. However, the initial excitement about applications of information theory to cognition and many other fields was quickly turned off by critics, including Shannon himself, who argued that the theory’s scope outside the realm of engineering is limited (Shannon, 1956; Luce, 2003). 

Yet, several decades later, we now see a renewed surge of interest in information theory, with numerous successful applications, ranging from theoretical neuroscience (e.g., Friston, 2010; Palmer et al., 2015; Tkacik & Bialek, 2016) to working memory (e.g., Bates et al., 2019; Jakob & Gershman, 2023), perception (e.g., Sims, 2016, 2018), decision making (e.g., Tishby & Polani, 2011; Lai & Gershman, 2021; Bhui et al., 2021), behavioral economics (e.g., Caplin et al., 2022; Prat-Carrabin & Woodford, 2022; Azeredo da Silveira et al., 2024), and language (e.g., Zaslavsky et al., 2018; Gibson et al., 2019), as well as broad applications in artificial intelligence (e.g., Tishby & Zaslavsky, 2015; Alemi et al., 2017; Du et al., 2020; Gualdoni et al., 2024). This recent body of work suggests that, in contrast to the initial reservations by Shannon and others, information theory provides a powerful framework for understanding both human and artificial intelligence at multiple levels of analysis (Marr, 1982). However, so far, information-theoretic approaches have been applied mostly independently in each area of cognitive science, without much interaction across disciplines.

The goal of this workshop is to create a multidisciplinary space for discussing the most recent advances at the intersection of information theory and cognitive science and to explore how this emerging research area can help the field advance toward a more comprehensive and principled mathematical theory of human cognition. In particular, we aim to attract and inspire intensive and meaningful dialogue between researchers who, while active in a gamut of fields, share the vision of communicating across the domain via concepts and ideas from information theory.

• Alemi, A. A., Fischer, I., Dillon, J. V., & Murphy, K. (2017). Deep variational information bottleneck. In International conference on learning representations.
• Azeredo da Silveira, R., Sung, Y., & Woodford, M. (2024, October). Optimally imprecise memory and biased forecasts. American Economic Review, 114(10), 3075–3118. doi: 10.1257/aer.20201806 
• Bates, C. J., Lerch, R. A., Sims, C. R., & Jacobs, R. A. (2019). Adaptive allocation of human visual working memory capacity during statistical and categorical learning. Journal of vision, 19(2), 11–11. 
• Bhui, R., Lai, L., & Gershman, S. J. (2021). Resource-rational decision making. Current Opinion in Behavioral Sciences, 41, 15–21. 
• Caplin, A., Dean, M., & Leahy, J. (2022). Rationally inattentive behavior: Characterizing and generalizing shannon entropy. Journal of Political Economy, 130(6), 1676-1715. doi: 10.1086/719276 
• Du, Y., Tiomkin, S., Kiciman, E., Polani, D., Abbeel, P., & Dragan, A. (2020). AvE: Assistance via empowerment. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, & H. Lin (Eds.), Advances in neural information processing systems (Vol. 33, pp. 4560–4571). Curran Associates, Inc. 
• Friston, K. (2010, Feb 01). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2), 127-138. 
• Gibson, E., Futrell, R., Piantadosi, S. P., Dautriche, I., Mahowald, K., Bergen, L., & Levy, R. (2019). How efficiency shapes human language. Trends in Cognitive Sciences, 23(5), 389-407. doi: https://doi.org/10.1016/j.tics.2019.02.003 
• Gualdoni, E., Tucker, M., Levy, R. P., & Zaslavsky, N. (2024). Bridging semantics and pragmatics in information-theoretic emergent communication. In The thirty-eighth Annual Conference on Neural Information Processing Systems. 
• Jakob, A. M., & Gershman, S. J. (2023, jul). Rate-distortion theory of neural coding and its implications for working memory. eLife, 12, e79450. doi: 10.7554/eLife.79450 
• Lai, L., & Gershman, S. J. (2021). Policy compression: An information bottleneck in action selection. In Psychology of learning and motivation. Psychology of Learning and Motivation.
• Luce, R. D. (2003). Whatever happened to information theory in psychology? Review of General Psychology, 7(2), 183-188. doi: 10.1037/1089-2680.7.2.183
• Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. New York, NY, USA: Henry Holt and Co., Inc. 
• Miller, G. A. (2003). The cognitive revolution: a historical perspective. Trends in Cognitive Sciences, 7(3), 141–144. 
• Palmer, S. E., Marre, O., Berry, M. J., & Bialek, W. (2015). Predictive information in a sensory population. Proceedings of the National Academy of Sciences, 112(22), 6908–6913. 
• Prat-Carrabin, A., & Woodford, M. (2022). Efficient coding of numbers explains decision bias and noise. Nature Human Behaviour, 6(8), 1142–1152. 
• Shannon, C. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379-423. 
• Shannon, C. (1956). The bandwagon. IRE Transactions on Information Theory, 2(1), 3-3. doi: 10.1109/TIT.1956.1056774 
• Sims, C. R. (2016). Rate–distortion theory and human perception. Cognition, 152, 181 - 198. doi: http://dx.doi.org/10.1016/j.cognition.2016.03.020 
• Sims, C. R. (2018). Efficient coding explains the universal law of generalization in human perception. Science, 360(6389), 652-656. 
• Tishby, N., & Polani, D. (2011). Information theory of decisions and actions. In V. Cutsuridis, A. Hussain, & J. G. Taylor (Eds.), Perception-action cycle: Models, architectures, and hardware (pp. 601–636). New York, NY: Springer New York. 
• Tishby, N., & Zaslavsky, N. (2015). Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw) (pp. 1–5). 
• Tkacik, G., & Bialek, W. (2016). Information ˇ processing in living systems. Annual Review of Condensed Matter Physics, 7(1), 89-117. doi: 10.1146/annurev-conmatphys-031214-014803 
• Zaslavsky, N., Kemp, C., Regier, T., & Tishby, N. (2018). Efficient compression in color naming and its evolution. Proceedings of the National Academy of Sciences, 115(31), 7937–7942.