Course ID:  M1801D8E

Credits: 3

Objective: How to measure information capacity How to compress information How to efficiently transmit information over noisy channels

Course Prerequisites: Probability and Random Process Signal and Systems

Outline:

1. Perfect Communication Thru a Noisy Channel Reliable transmission of data The binary symmetric channel Error correcting strategies: repetition Shannon`s Noisy Channel Coding Theorem Definition of Entropy/Mutual information Measuring the Information Content 

2. Source Coding -- Data Compression Why Entropy Shannon`s Source Coding Theorem Arithmetic coding 

3. Noisy Channel Coding Theorem Review of inference Mutual information The data processing theo

Teaching Method: Lectures and in class Discussion

Reference:  

1.Cover/Thomas, "Element of Information Theory", 1991. 

2.David MacKay, "Information Theory, Inference, and Learning Algorithms", 2003. 

3.Robert Gray, "Entropy and Information Theory", Springer-Verlag 2013.