EE 376A: Information Theory

Stanford University

Winter 2009-2010

Claude Elwood Shannon
A short movie about Shannon

Announcements

Reminder: final review session on March 16th, Tuesday, at Gates B12, 4:00pm -- 5:00pm
[2010/03/12]Sample finals posted.
[2010/03/10]The due day for HW8 is extended to Mar. 15th.
[2010/02/07]Sample midterms posted.
HW1 posted
[2010/01/05] Please make an account at EEClass if you are enrolled. We are going to use that to keep all the grades.
[2010/01/01] The first lecture will be on Tuesday, January 12th , i.e., we will start on the second week of the quarter.
[2010/01/01] Lectures will be made available in video via the ViewXtreme project.

Course Description

We will cover the following topics in class:

Information measures and their properties
Fixed length lossless compression
The Asymptotic Equipartition Property
Variable length lossless compression
Channel coding and capacity
Lossy source coding and the rate distortion function
Method of types
Joint source-channel coding

Prerequisites:

You need to be comfortable with undergraduate probability. Beyond that, motivation and maturity to cope with some abstract notions.

Course Reader:

The lectures will follow the course reader

Textbooks:

The main recommended textbook is Elements of Information Theory by Cover and Thomas, 2nd Edition, New York, Wiley, 2006. Additional texts include:

R. B. Ash, Information Theory, Dover, 1990.
T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression, Prentice Hall, 1971.
I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete
Memoryless Systems, Academic Press, 1981.
R.G. Gallager, Information Theory and Reliable Communication,Wiley, 1968.

Course Info

Times and Location:

HEWLETT 201, Tue&Th 11AM-12:15pm, 3 units Email: ee376a-win0910-staff@lists.stanford.edu

Teaching staff:

Instructor: Tsachy Weissman

Office: Packard 256

Tel: 736-1418

Email: tsachy *AT* stanford.edu

Office hours: Th 13:30 -- 15:30

Teaching Assistant: Lei Zhao

Office: Packard 251

Tel: 723-4544

Email: leiz *AT* stanford.edu

Office hours: Mon. 4:00pm -- 5:00pm Packard 107

Teaching Assistant: Rajiv Agarwal

Office: Packard 360

Email: rajivag *AT* stanford.edu

Office hours: W 7:30-8:30 pm, Packard 106

Administrative Associate: Kelly Yilmaz

Office: Packard 259

Tel: 723-4539

Email: yilmaz *AT* stanford.edu

Course Requirement

Grading will be based on homework sets (30%) (approximately weekly), a midterm (30%), and a final(40%) .
Midterm: Feb 11th, 5:30pm -- 7:30 pm, Location: Braun Auditorium in the Mudd Chemistry building.
Final: Mar. 18th, 3:30pm -- 6:30 pm, Location: Hewlett 201.
Class participation is required.

Last modified: