Instructor: Shashi Prabh
Email: shashi.prabh@snu
Office: D036E
Office hour: 11:00 AM - 1:00 PM Fri, or by appointment
Lectures: 2:00 - 3:00 PM Tue, Thu, Fri
Location: B108
Knowledge of probability theory covered in a typical introductory course is necessary. The relevant topics of probability theory will be taught in the course but at a fast pace. This course is suited for those who are not averse to mathematics. Students intending to register for the course are expected to meet the instructor beforehand to assess whether they can handle the course material.
Review of combinatorics, sets and probability theory. Concept of information, entropy, entropy rate, source coding, data compression algorithms, noisy channel coding, channel capacity, Gaussian channels, multiple access channels, broadcast channels and Kolmogorov complexity.
Thomas M. Cover, Joy A. Thomas. Elements of Information Theory, 2nd Edition, Wiley 2006. ISBN: 978-0-471-24195-9
David Applebaum. Probability and Information: An Integrated Approach, 2nd Edition, Cambridge University Press, 2008. ISBN-10: 0521899044
David MacKay. Information Theory, Inference and Learning Algorithms, Cambridge University Press, 2003. ISBN-10: 0521642981. Electronic version is available here.
The book by Cover and Thomas will be the main textbook. This book is primarily targeted towards advanced UG and graduate students. Students finding difficulties should consult the other two books. Applebaum's book is a very good introductory text that also provides adequate background in probability theory. Mackay's book is excellent and very readable. Students interested in machine learning should definitely read it.
Assignments and Quizzes: 15 %
Mid-term exam: 35 %
Final exam: 35 %
Project: 15 %