OE3E15: Information Theory and Coding

Course Content: 

Review of probability theory, Entropy: marginal entropy, joint entropy, conditional entropy, and the chain rule for entropy. Mutual information between ensembles of random variables. 

 Source Coding theorems: prefix, variable, and fixed-length codes. Channel models and channel capacity. Channel Coding theorem. 

Linear Block Codes: Generator and parity check matrices, Minimum Distance, Syndrome decoding, Bounds on minimum distance. 

Cyclic Codes: Finite Fields, binary BCH codes, RS Codes.

Convolutional Codes: Encoders, Trellis, Viterbi decoding.             

Recommended Books:

• • • Thomas M. Grover and Joy A. Thomas, “Elements of Information Theory,” Wiley. 

    • Shu Lin and Daniel J. Costello, Jr., Error Control Coding: Fundamentals and Applications, 2nd Ed., Prentice-Hall, 2004.

    • B. Sklar, Digital Communications: Fundamentals and Applications, 2nd Ed., Prentice-Hall, 2001.

    • John G. Proakis and Masoud Salehi, Digital Communications, 5e, McGraw-Hill Education, 2007.