created by Wim van Dam on 14 December 2007 • last updated on 15 March 2008
Announcements:
[March 15] The Final Paper is due (by email to WvD) on Wednesday, March 19 at 11:59 pm. In the meantime WvD will be traveling with limited email access.
[March 5] Four papers from a previous installment of the course have been posted; all of these are examples of well written papers (although they are -of course- not perfect).
[March 5] Some notes and advice on how to write the Final Paper have been posted.
[Feb 29] Exercise 3 has been posted, which is due Monday March 10 at 4 pm.
[Feb 29] The slides of Weeks 5, 6, 7 and 8 have been posted.
[Feb 25] The Final Paper will be due Wednesday, March 19.
[Feb 13] The handout on the noise channel theorem has been posted.
[Feb 8] Please email me the following information regarding our paper: title, abstract, and projected references (you will need at least two besides Cover and Thomas).
List of topics: Entropy, mutual information, and Shannon's coding theorems, lossless source coding, Huffman, Shannon-Fano-Elias, and arithmetic codes, channel capacity; aspects of rate-distortion theory, lossy source coding, source-channel coding, Kolmogorov/algorithmic complexity and information; applications of information theory in various fields
This 4 unit, graduate course is listed both for the CS department and the ECE department, and will be taught alternately by Wim van Dam (CS) and Kenneth Rose (ECE). For CS students this course will satisfy the 'theory track'.
Prerequisites:
ECE 140 or PSTAT 120A-B
Required Texts:
Elements of Information Theory, Thomas M. Cover and Joy A.Thomas, John Wiley, 1991
Both editions, the 1st from 1991 or the 2nd from 2006, are fine.
Final Grade:
The final grade will be determined 50% on the basis of the take home exercises and 50% on the basis of the final paper that you will write.
Weekly Schedule:
Monday 11:00-13:00: Class in Phelps 1401
Tuesday 11:00-12:00: Office hours WvD in Harold Frank Hall, room 2151
Wednesday 11:00-13:00: Class in Phelps 1401
Contacting/Questions:
I prefer that you use my office hours for your questions, rather than doing a lengthy Q&A exchange via email or an impromptu office visit. Please email me for an appointment in the afternoon if the office hours do not work for you. Thanks.