EC304 Information Theory and Coding Techniques

Title: Information Theory and Coding Techniques

Semester: Sixth Semester B Tech (ECE)

3-0-0, 3 Credits

Prerequisites: probability theory

Information measures, Asymptotic Equipartition Property (AEP), Characterization of Information Sources. Entropy rate of a stochastic process, Introduction to Lossless Data Compression (Source Coding) for discrete sources, Shannon’s noiseless source coding theorem, Kraft-McMillan inequality for prefix-free codes, Shannon-Fano-Elias code and Huffman codes. Universal source coding - Arithmetic coding and Lempel-Ziv coding. Introduction to Rate Distortion theory. Discrete channel characterization, channel capacity, Shannon’s noisy-channel coding theorem, reliability exponents. Differential Entropy, Introduction to the Gaussian Channel. Introduction to Error Control Codes – Hamming Code and Linear Block Codes.

Text Books/References:

1. Thomas M. Cover and Joy A. Thomas, “Elements of Information Theory”, 2^nd Edition, Wiley Interscience, 2006.

2. David MacKay, “Information Theory, Inference, and Learning Algorithms”, Cambridge University Press, 2003. (Free legal version of online copy of the book is available at http://www.inference.phy.cam.ac.uk/mackay/itila/book.html).

3. Robert B. Ash, “Information Theory (Dover Books on Mathematics)”, Dover Publications, 1990.

4. Claude E. Shannon, “A Mathematical Theory of Communication”, Bell Sys. Tec. Journal, vol. 27, pp. 379–423, 623-656, 1948.