Program Elective -Information Theory and Coding

Subject Name: Information Theory and Coding

Subject Code : PE-EC505A (Program Elective I)

Credit-3

3L/Week

Course Instructor : Prof.(Dr.) Tribeni Prasad Banerjee

Typical Class Schedule:

Wednesday :12:00-01:00 PM

Saturday: 9:00-10:00 AM

Sunday : 12:00-01:00 PM

Syllabus: [Model Curriculum of  AICTE & New Guidelines from 2018]

Basics of information theory, entropy for discrete ensembles; Shannon's noiseless coding theorem; Encoding of discrete sources.

Markov sources; Shannon's noisy coding theorem and converse for discrete channels;Calculation of channel capacity and bounds for discrete channels; Application to continuous channels.

Techniques of coding and decoding; Huffman codes and uniquely detectable codes;Cyclic codes, convolutional arithmetic codes.

The Course Outcomes:

At the end of the course, students will demonstrate the ability to:

1.Understand the concept of information and entropy

2.Understand Shannon’s theorem for coding

3.Calculation of channel capacity

4.Apply coding techniques

Text/Reference Books:

1.N. Abramson, Information and Coding, McGraw Hill, 1963.

2.M. Mansurpur, Introduction to Information Theory, McGraw Hill, 1987.

3.R.B. Ash, Information Theory, Prentice Hall, 1970.

4.Shu Lin and D.J. Costello Jr., Error Control Coding, Prentice Hall, 1983.

Web resource:

1. Course Page [EC604B]

2. Information Theory 1-Lecture 1 of  Series on Digital Communication by Prof.Bikash. Kumar. Dey , Department of Electrical Engineering,IIT Bombay.

3.Information Theory II-Lecture 1 of  Series on Digital Communication by Prof.Bikash. Kumar. Dey , Department of Electrical Engineering,IIT Bombay.

4. Lecture Notes by Prof. em. J. L. Massey

5. Information Theory, Coding and Cryptography. Instructor: Dr. Ranjan Bose, Department of Electrical Engineering, IIT Delhi.

6. Dr. Andrew Thangaraj, Department of Electronics & Communication Engineering, IIT Madras.

7. Lecture notes on Information Theory

8. Lecture notes of Statistics by John Duchi

9. Digital Communications and Signal Processing by K. Vasudevan, IIT Kanpur.

You may want to read more at the following wikis:

1. Coding Theory Wiki 

2. Error-correcting codes Wiki 

3. Information Theory Wiki 

Read about Hamming at Richard Hamming 

Read about Shannon at Claude E. Shannon 

Read general info about coding theory at Coding theory: first 50 years

You can see the IEEE resource for more information.

Assessment Process:

Class Attendance (25%) + Assignment (25%)+ Project (Mini) (50%)

Assignments 1