I am going to be at a conference in Berlin on Aug. 27, so there will be no class that day.  The first day of class will be Sep. 3.  Students should get the Pierce book and read chapters 1 through 5, and for the first class I'll give an introductory lecture on probability and information theory.  See you in September!

Week 1: Introduction

Pierce, J. R. 1980.  An Introduction to Information Theory: Symbols, Signals, and Noise.  2nd edition. New York: Dover. 

Slides part A and part B

Week 2: Early linguistic applications

Shannon, C. E. 1951. "Prediction and entropy of printed English." Bell Systems Technical Journal 30: 50–65.  

Brown, P. F., S. A. Della Pietra, V. J. Della Pietra, J. C. Lai, and R. L. Mercer. 1992. "An estimate of an upper bound for the entropy of English." Computational Linguistics 18: 31-40.

Shannon game simulation.

Week 3: Phonology I: Distinctive features

Cherry, E. C., M. Halle, and R. Jakobson. 1953. "Toward the logical description of languages in their phonemic aspect.Language 29: 34-46.

Hume, E. and F. Mailhot. 2011. Distinctive features and information theory. Presentation given at the Laboratoire de Phonétique et Phonologie. Paris. March 14, 2011.

Hume, E., K. Hall, A. Wedel, A. Ussishkin, C. Gendrot, and M. Adda-Decker. 2011. "Anti-markedness patterns in French epenthesis: An information-theoretic approach." In Proceedings of the 37th Annual Meeting of the Berkeley Linguistics Society (BLS 37), pp. 104-123.

Week 4: Phonology I: Distinctive features (cont.)

Goldsmith, J. and J. Riggle. 2012. "Information theoretic approaches to phonological structure: the case of Finnish vowel harmony.Natural Language and Linguistic Theory 30: 859-896.

Week 5: Phonology II: Functional load 

Hockett, C. F. 1967. "The quantification of functional load." Word 23: 320–339.

Surendran, D., and P. Niyogi. 2006. "Quantifying the functional load of phonemic oppositions, distinctive features, and suprasegmentals." In O. Nedergaard Thomsen (ed.), Competing Models of Language Change: Evolution and Beyond, 43-58. Amsterdam and Philadelphia: John Benjamins.

Surendran, D., and G. Levow. 2004. "The functional load of tone in Mandarin is as high as that of vowels." In Speech Prosody 2004, 99–102.

Week 6: Phonology II: Functional load (cont.)

Hume, E. and F. Mailhot. 2013. "The role of entropy and surprisal in phonologization and language change." In A. Yu, (ed.), Origins of Sound Patterns: Approaches to Phonologization. Oxford University Press.

Wedel, A., A. Kaplan, and S. Jackson. 2013. "High functional load inhibits phonological contrast loss: A corpus study." Cognition 128: 179–186.

Wedel, A., S. Jackson, and A. Kaplan. 2013. "Functional load and the lexicon: Evidence that syntactic category and frequency relationships in minimal lemma pairs predict the loss of phoneme contrasts in language change." Language and Speech 56:395-417.

Week 7: Entropy Rate

Aylett, M.P. and A. Turk. 2004.  "The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech." Language and Speech 47.1.31–56

Pellegrino, F., C. Coupé, and E. Marsico. 2011. "A cross-language perspective on speech information rate." Language 87: 539–558.

Week 8: Entropy Rate (cont.)

Genzel, D., and E. Charniak. 2002. "Entropy rate constancy in text." In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 199–206.

Genzel, D., and E. Charniak. 2003. "Variation of entropy and parse trees of sentences as a function of the sentence number." In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, 65–72.

Levy, R., and T. F. Jaeger. 2007. "Speakers optimize information density through syntactic reduction." In Advances in Neural Information Processing Systems (NIPS) 19, 849–856. 

Some experiments in Python

Week 9: Entropy Rate (cont.)

Frank, A. F., and T. F. Jaeger. 2008. "Speaking rationally: Uniform information density as an optimal strategy for language production." In The 30th Annual Meeting of the Cognitive Science Society, 933–938.

Jaeger, T. F. 2010. "Redundancy and reduction: Speakers manage syntactic information density." Cognitive Psychology 61: 23–62. 

Week 10: Lexicon

Piantadosi, S. T., H. Tily, and E. Gibson. 2009. "The communicative lexicon hypothesis." The 31st Annual Meeting of the Cognitive Science Society, 2582–2587.

Piantadosi, S. T., H. Tily, and E. Gibson. 2011. "Word lengths are optimized for efficient communication." Proceedings of the National Academy of Sciences, 3526–3529. 

Piantadosi, S. T., H. Tily, and E. Gibson. 2012. "The communicative function of ambiguity in language." Cognition 122: 280–291.

Week 11: Origins of Language

Nowak, M.A., and D.C. Krakauer. 1999. "The evolution of language." Proceedings of the National Academy of Sciences 96:8028–8033.

Plotkin, J.B., and M.A. Nowak. 2000. "Language evolution and information theory." Journal of Theoretical Biology 205:147–159.

Nowak, M.A., and N.L. Komarova. 2001. "Towards an evolutionary theory of language." Trends in Cognitive Sciences 5:288–295.

Week 12: Language Processing

Hale, J.. 2001. "A probabilistic Earley Parser as a psycholinguistic model." In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies, 1–8.

Boston, M. F., Hale, J., Kliegl, R., Patil, U. & Vasishth, S. 2008. "Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus." Journal of Eye Movement Research, 2(1):1, 1-12.

N. J. Smith and R. Levy. 2013. "The effect of word predictability on reading time is logarithmic." Cognition 128(3):302-319.

Week 13: Language Complexity

Juola, P. 1998. "Measuring linguistic complexity: The morphological tier." Journal of Quantitative Linguistics 5(3):206-213.

Juloa, P. 2007. "Assessing linguistic complexity." In M. Miestamo, K. Sinnemaki and F. Karlsson (eds.) Language Complexity: Typology, Contact, Change. Amsterdam : John Benjamins Press.

Moscoso del Prado Martín, F. 2011. "The mirage of morphological complexity." In P
roceedings of the 33rd Annual Conference of the Cognitive Science Society. Austin, TX.

Week 14: Morphology

Ackerman, F., J. P. Blevins, and R. Malouf. 2009. "Parts and wholes: Patterns of relatedness in complex morphological systems and why they matter." In James P. Blevins and Juliette Blevins (ed.), Analogy in Grammar: Form and Acquisition. Oxford University Press, Oxford. Pages 54-82. 

Ackerman, F. and R. Malouf. 2013. "Morphological organization: The Low Conditional Entropy Conjecture." Language 89:429-464 .

Week 15: Presentations