Fall 2026 Course Topics and Readings
Suggested textbook sections are listed by topic, in the order of the lecture plan below. The readings are organized by topic rather than lecture date because the schedule is tentative.
Section numbers refer to Gallager (1st edition), Vershynin (1st edition), and Understanding Machine Learning (UML). Related readings provide background rather than full coverage. Where no direct textbook section is listed, use the lecture material, including recordings in Canvas and notes on Ed.
Inference; Hypothesis Testing; MAP and ML
Gallager, §§8.1–8.2 Gaussian Hypothesis Testing; Error Probability
Gallager, §§8.2.2–8.2.3 (Gaussian detection), §8.4 (error curves) Minimum-Distance Decoding
Related reading: Gallager, §8.5 (multiple-hypothesis detection); see lecture material for minimum-distance decoding Machine Learning; Empirical Risk Minimization
UML, §§2.1–2.3 Weak LLN
Gallager, §1.7.1 Markov's Inequality; Chebyshev's Inequality
Gallager, §§1.6.1–1.6.2; Vershynin, §1.2 CLT
Gallager, §1.7.3; Vershynin, §1.3 Hoeffding's Inequality
Vershynin, §2.2; UML, Appendix B.4 Uniform Convergence; Generalization Bounds
UML, §2.3.1 and §§4.1–4.2 Random Vectors; Covariance Matrices; Linear Transforms of Random Vectors
Related reading: Vershynin, §3.2 (covariance and linear transformations); Gallager, §§3.3–3.4 (Gaussian examples and covariance) Gaussian Random Vectors
Gallager, §§3.3–3.4; Vershynin, §3.3.2 MMSE Estimation; Geometric Interpretation; Orthogonality Principle
Gallager, §10.1.1 (squared-error estimation), §10.6 (geometry and orthogonality) Linear MMSE Estimation
Gallager, §10.3 Principal Component Analysis; Optimal Linear Approximation
UML, §23.1; Vershynin, §3.2.1 (PCA background) Gaussian Conditioning
Gallager, §3.5 Gaussian Estimation
Gallager, §10.2 Innovations
Related reading: Gallager, §§10.2.1, 10.4.3 (iterative estimation); see lecture material for innovations Kalman Filtering
Gallager, §10.2.2 (scalar), §10.4.4 (vector) Random Processes; Stationarity
Gallager, §1.4 and the opening of §3.6 (random processes), §3.6.1 (stationarity) Power Spectral Density; Linear Systems with Random Process Inputs
Gallager, §§3.6.6–3.6.7 Cross-Correlation; Cross-Spectral Density
Lecture material White Noise
Gallager, §3.6.8 (white Gaussian noise) Linear Estimation
Gallager, §10.3 (finite-vector estimation) Wiener Filtering; Spectral Factorization; Causal Wiener Filtering
Lecture material Lookahead; Linear Prediction; Prediction in Practice
Lecture material
Fall 2026 Tentative Lecture Plan
Lecture topics, pacing, and teaching assignments are tentative and subject to change. Lectures meet on Tuesdays and Thursdays. Lecture numbers include the midterms and Democracy Day. University holidays and recess dates follow the Stanford academic calendar.
Lecture 1 (September 22): Inference; Hypothesis Testing; MAP and ML — Professor Ayfer Özgür Lecture 2 (September 24): Gaussian Hypothesis Testing; Error Probability; Minimum-Distance Decoding — Professor Ayfer Özgür Lecture 3 (September 29): Machine Learning; Empirical Risk Minimization; Weak LLN — Professor Ayfer Özgür Lecture 4 (October 1): Markov's Inequality; Chebyshev's Inequality; CLT; Hoeffding's Inequality; Uniform Convergence — Professor Ayfer Özgür Lecture 5 (October 6): Uniform Convergence; Generalization Bounds; Random Vectors; Covariance Matrices; Linear Transforms of Random Vectors — Professor Ayfer Özgür Lecture 6 (October 8): Gaussian Random Vectors — Professor Ayfer Özgür Lecture 7 (October 13): MMSE Estimation; Geometric Interpretation; Orthogonality Principle — Professor Ayfer Özgür Lecture 8 (October 15): Midterm I (in class during class time; covers the first three weeks) Lecture 9 (October 20): Linear MMSE Estimation; Principal Component Analysis; Optimal Linear Approximation — Professor Tsachy Weissman Lecture 10 (October 22): Gaussian Conditioning; Gaussian Estimation; Innovations — Professor Tsachy Weissman Lecture 11 (October 27): Kalman Filtering — Professor Tsachy Weissman Lecture 12 (October 29): Random Processes — Professor Tsachy Weissman Lecture 13 (November 3): Democracy Day (no classes) Lecture 14 (November 5): Midterm II (in class during class time) Lecture 15 (November 10): Stationarity; Power Spectral Density — Professor Tsachy Weissman Lecture 16 (November 12): Linear Systems with Random Process Inputs; Cross-Correlation; Cross-Spectral Density — Professor Tsachy Weissman Lecture 17 (November 17): White Noise; Linear Estimation; Wiener Filtering — Professor Tsachy Weissman Lecture 18 (November 19): Spectral Factorization; Causal Wiener Filtering — Professor Tsachy Weissman Lecture 19 (December 1): Lookahead; Linear Prediction — Professor Tsachy Weissman Lecture 20 (December 3): Course Wrap-Up and Prediction in Practice — Professor Ayfer Özgür
|