This is the site of Wim van Dam's graduate course "Information Theory" (CS225 and ECE205A) at UC Santa Barbara in Spring 2014.
Copyright 2014 Wim van Dam (UC Santa Barbara). It is forbidden to copy, distribute, or transit this work, nor is it allowed to alter, transform, or build upon this work.
- 2014-05-01: The course schedule, the slides, and the HW Answers have moved to GauchoSpace.
- 2014-04-30: HW3 has been posted on GauchoSpace. Your answers are due 2014-05-07, at 23:59 and should be submitted using GauchoSpace as well.
- 2014-04-27: The HW1 and HW2 scores and the answers to HW2 have been posted on GauchoSpace.
- 2013-11-01: Welcome to the CS225|ECE205A web site for Spring 2014, UCSB.
General Course Information
- Course no: CS225 and ECE205A
- Course title: Information Theory
- List of topics: Entropy, mutual information, and Shannon's coding theorems, lossless source coding, Huffman, Shannon-Fano-Elias, and arithmetic codes, channel capacity; aspects of rate-distortion theory, lossy source coding, source-channel coding, Kolmogorov/algorithmic complexity and information; applications of information theory in various fields
- Credit: This 4 unit, graduate course is listed both for the CS department and the ECE department, and will be taught alternately by Wim van Dam (CS) and Kenneth Rose (ECE). For CS students this course will satisfy the 'theory track'.
- Prerequisites: ECE 140 or PSTAT 120A-B
Professor
- Wim van Dam
- vandam@cs.____.___
- Harold Frank Hall, Room 2151
Reader
- Li Zheng
- ucsbcs225@gmail.___
Required Texts
- Elements of Information Theory, Thomas M. Cover and Joy A.Thomas, John Wiley, 1991
- Both editions, the 1st from 1991 or the 2nd from 2006, are fine.
Final Grade
- Your final grade for the course will be determined by your scores for the 5 homework assignments. Each assignment will have 4 questions, worth 3 points each, hence in total you can get 60 points.
- Per question the 3 points are awarded using the following scale:
- 3 points: Exemplary; student applies knowledge with virtually no conceptual or procedural errors
- 2 points: Adequate; student applies knowledge with no significant conceptual errors and only minor procedural errors.
- 1 point: Minimal; student applies knowledge with occasional conceptual errors and only minor procedural errors.
- 0 points: Unsatisfactory; student makes significant conceptual and/or procedural errors when applying knowledge.
LaTeX
It is strongly recommended that you typeset your answers to the homework assignments in LaTeX. See here for information about LaTeX and here for some general advice on writing academic papers.
Class and Office Hours
- Monday 11:00—12:50: Class in Phelps 2510
- Monday 13:00—15:00: Office hours in HFH 2151
- Wednesday 11:00—12:50: Class in Phelps 2510
The norm will be that classes lasts for 90 minutes, but sometimes the amount of material might require us to run the full 110 minutes.
Contacting/Questions
- I prefer that you use my office hours for your questions, rather than doing a lengthy Q&A exchange via email or an impromptu office visit. Please email me for an appointment in the afternoon if the office hours do not work for you. Thanks.
Academic Honesty
The following applies to every course you attend at UC Santa Barbara (from UCSB Campus Regulations, Chapter VII: "Student Conduct and Discipline"):
It is expected that students attending the University of California understand and subscribe to the ideal of academic integrity, and are willing to bear individual responsibility for their work. Any work (written or otherwise) submitted to fulfill an academic requirement must represent a student’s original work. Any act of academic dishonesty, such as cheating or plagiarism, will subject a person to University disciplinary action. Using or attempting to use materials, information, study aids, or commercial “research” services not authorized by the instructor of the course constitutes cheating. Representing the words, ideas, or concepts of another person without appropriate attribution is plagiarism. Whenever another person’s written work is utilized, whether it be a single phrase or longer, quotation marks must be used and sources cited. Paraphrasing another’s work, i.e., borrowing the ideas or concepts and putting them into one’s “own” words, must also be acknowledged. Although a person’s state of mind and intention will be considered in determining the University response to an act of academic dishonesty, this in no way lessens the responsibility of the student.
Specifically for the current CS225|ECE205A course this means that
- You are not allowed to copy or transcribe answers to homework assignments from others or other sources.
- Although you are allowed to discuss homework assignments with others, you should write down your answers independently. You should always be able to argue and explain your answers when asked for clarifications.
- When you will be unable to hand in the homework in time you should report this to WvD as soon as possible, but always before the deadline. No matter the reason, you will always be asked to present documentation.
- When in doubt, ask.
- Students violating the rules of Academic Honesty will receive an "F" for the course and will be reported to the Dean of Students Office.
Schedule of Topics and Homeworks
Homework
- HW1: "Entropy"; announced 2014-04-09 (Week II, Wednesday); due 2014-04-16 (Week III, Wednesday, 23:59)
- HW2: "Data Compression"; announced 2014-04-16 (Week III, Wednesday); due 2014-04-23 (Week IV, Wednesday, 23:59)
- HW3: "AEP, Gambling"; announced 2014-04-30 (Week V, Wednesday); due 2014-05-07 (Week VI, Wednesday, 23:59)
- HW4: "Kolmogorov Complexity, Channel Capacity"; announced 2014-05-14 (Week VII, Wednesday); due 2014-05-21 (Week VIII, Wednesday, 23:59)
- HW5: "Error Correcting Codes, Rate-Distortion Theory"; announced 2014-05-28 (Week IX, Wednesday); due 2014-06-04 (Week X, Wednesday, 23:59)