Jan-Apr 2023
ECS 426/626: Information Theory and Coding
Pre-requisites: Probability theory, basic knowledge of digital communication systems
Teaching assistants: NONE
Course syllabus:
Module 1: ELEMENTS OF INFORMATION THEORY AND SOURCE CODING: Introduction, information as a measure of uncertainty, Entropy and its properties, Discrete memoryless channels, Mutual information and its properties, BSC, BEC. Channel capacity, Shannon’s Noiseless coding, Shannon’s theorem on coding for memoryless noisy channels, , Theorem of decodability, Average length of encoded message, Shannon’s binary encoding, Fundamental theorem of discrete noiseless coding, Huffman’s minimum redundancy codes.
Module 2: LINEAR BLOCK CODES: Groups and fields, Galois fields and its construction, Minimal polynomial, Vector spaces, Dual spaces, Linear block codes, Syndrome and error detections, Minimum distance, Error detecting and correcting capabilities of a block code, Standard array and Syndrome decoding, Hamming code.
Module 3: CYCLIC CODES: Description of cyclic codes, Polynomial representation, Minimum degree code polynomial, Generator polynomial, Generator matrix, Systematic form, Parity check matrices, cyclic codes encoders, Syndrome computation and error detection, Cyclic Hamming codes, Decoding of cyclic codes.
Module 4: CONVOLUTIONAL CODES: Encoding of Convolution codes, Structural properties of Convolution codes, State diagram, Code tree, Trellis diagram, Free distance, Coding gain, Viterbi decoding, distance properties of binary convolutional codes.
Reference Books:
Thomas Cover and Joy Thomas, Elements of Information Theory, John Wiley and Sons.
Simon Haykin, Digital Communication Systems, Wiley India.
3. Shu Lin and J. Costello, Error Control Coding, 3rd Ed., Prentice Hall.
4. Ranjan Bose, Information Theory, Coding and Cryptography, 2nd Ed. McGraw Hill
5. David McKay, Information Theory, Pattern recognition and neural networks, Cambridge University Press.
Class time:
I SLOT: Monday 12 PM to 12:55 PM, Thursday: 2 PM to 2:55 PM, Friday 12 PM to 12:55 PM
Grading policy:
TBA
Homeworks:
Announced on Classroom portal