M/W 4pm to 5:20pm, 4025 CIF | Please refresh the page twice to make sure you are seeing the latest updates
First day information
Homework and MPs
Fall 2025: Midterm Information
Fall 2025: Final Exam Information 🔥
Tentative course calendar:
🟢 = must read. 🔵 = additional reading.
| Lecture |
Topics |
Lecture material (web folder) |
Reading |
Homework |
MP |
| 1. Mon |
|
|
|
|
|
| Aug 25 |
Course logistics + big picture |
|
|
|
|
- Generative models overview | - lec1a_slides_overview.pptx
- lec1_video
| 🔵 Gen AI overview youtube (Stanford)
🔵 History of diffusion youtube (Yang Song) | ✍️ HW 1
(foundations) | |
| 2. Wed
Aug 27 | Review1: Probability
- Bayes, MLE, Multivariate, …
- Conditional independence, Markov
- Expectation Maximization (EM) | - lec2a_slides_prob_EM1.pptx
- lec2b_annotated.pdf
- lec2c_notes_EM.pdf
- Lec2_video | 🔵 Probability review (online book) | | |
| 3. Wed
Sep 3 | Review2: Probability, Linear Algebra, DL
- EM continued
- PCA, SVD, …
- Basic deep learning | - lec3a_slides_EM2_Eigen.pptx
- lec3b_annotated.pdf
- lec3c_notes_Eigen.pdf
- lec3_video | 🔵 Gilbert Strang’s lectures on null space, rank, etc.
🔵 EM tutorial by D. Lin PDF
🔵 SVD video here | | |
| 4. Mon
Sep 8 | Variational Inference, VAE
- AutoEncoder (AE)
- Semantics
- VAE (moon eclipse), ELBO | - lec4a_notes_PCA.pdf
- lec4b_slides_AE_VAE.pptx
- lec4_video | 🟢 Intro. to VAEs (Kingma): chap. 1,2 | | 🖥️ MP 1 |
| 5. Wed
Sep 10 | VAE visualization and problems
- Entropy push pull
- Prior hole
- Posterior collapse | - lec5a_slides_VAE_ELBO.pptx
- lec5b_annotated.pdf
- lec5_video | 🟢 Prior hole paper (UIUC) PDF
🔵 IMUV paper (UIUC)
🔵 VQ-VAE paper (NeuRIPS)
🔵 Learning latent prior paper (UCLA) PDF | 🚫 HW 1
| |
| 6. Mon
Sep 15 | Diffusion Models 1
- Hierarchical VAEs
- Variational diffusion models (VDM) | - lec6a_slides_Diffusion1.pptx
- lec6b_annotated.pdf
- lec6_video | 🟢 Luo’s tutorial, chapters 1-2 | | |
| 7. Wed
Sep 17 | Diffusion Models 2
- Gaussian encoders
- Updated ELBO | - lec7a_slides_Diffusion2.pptx
- lec7b_annotated.pdf
- lec7_video | 🟢 Luo’s tutorial, chapters 3-4
🔵 Stanley Chan tutorial | | |
| 8. Mon
Sep 22 | Diffusion Models 3
- 3 different interpretations
- Image prediction
- Noise prediction
- Tweedie’s formula → score matching | - lec8a_slides_diffusion3.pptx
- lec8b_annotated.pdf
- lec8_video | | | |
| Wed
Sep 24 | No class | | | | |
| 9. Mon
Sep 29 | Diffusion Models 4
- Score as a MMSE estimator
- Langevin dynamics
- Noise annealing
| - lec9a_slides_score.pptx
- lec9b_annotated.pdf
- lec9_video | 🟢 Grad. of data distribution (Song) blog
🟢 Score Matching (Helsinki) PDF
🔵 Elad’s lecture 3 video youtube
🔵 Learning from thermodynamics PDF
| ✍️ HW2
(EM, GMM, VAE) | 🚫 MP1 |
| 10. Wed
Oct 1 | Diffusion Models 5
- Zoom out / big picture
- Annealed Langevin dynamics
- Connect the dots: score, denoising, DDPM
- Time varying force field = differential equations | - lec10a_bigpicture.pptx
- lec10b_annotated.pdf
- lec10_video | 🟢 DDPM paper (Berkeley) PDF
🔵 Improved DDPM paper (OpenAI) PDF
🔵 DDIM paper (Stanford) paper PDF
🔵 Diffusion language models video
| | |
| 11. Mon Oct 6 | Guidance
- Classifier based guidance
- Derivative of logits
- Classifier free guidance
- Scaling factor $s$ | - lec11a_slides_guidance.pptx
- lec11b_annotated.pdf
- lec11_video | 🟢 GLIDE T2I models (OpenAI) PDF
🔵 Classifier free guidance (OpenAI) PDF
| | |
| 12. Wed Oct 8 | CLIP and Text-to-Image (T2I) Models
- T2I problem statement
- CLIP
- Latent Diffusion Model (LDM) | -lec12a_slides_FID-CLIP.pptx
- lec12b_annotated.pdf
- lec12_video | 🟢 CLIP: Visual from language PDF
🔵 Latent Diffusion paper (Heidelberg) | 🚫 HW 2 | 🖥️ MP 2 |
| 13. Mon
Oct 14 | Inverse problems + Posterior sampling
- IP problem statement
- Approximate conditional score
- Diffusion based IP
- DPS | - lec13a_slides_inverse.pptx
- lec13b_annotated.pdf
- lec13_video | 🟢 Diffusion posterior sampling (DPS) PDF
| | |
| 14. Wed
Oct 15 | DPS and Pi-GDM
- Approximating likelihood with single sample
- Approximating likelihood with Gaussian
- Colorization
- Deblurring
- Inpainting
| - lec14a_DPS_PiGDM.pptx
- lec14b_annotated.pdf
- lec14_video | 🟢 RePaint and inpaint paper (ETH)
🟢 Pi-GDM (NVidia) paper | | |
| 15. Mon
Oct 20 | Applications of Inverse Problems
- Source (speech) separation
- Motion tracking and mapping
- Inverse path planning | - lec15a_slides_inverse_apps.pptx
- lec15b_annotated.pdf
- lec15_video | 🟢 ArrayDPS paper (UIUC)
🟢 Image super resolution paper (Google)
🔵 Pose estimation paper (UIUC)
🔵 Inverse path planning paper (UIUC) | ✍️ HW 3
(diffusion and mid-term practice) | |
| 16. Wed
Oct 22 | Mid-term review | - All slides
- lec16_video | 🔥 See mid-term related information
on top of this webpage | | |
| 17. Mon
Oct 27 | Mid-term review | - All slides
- lec17_video | | | |
| 18. Wed,
Oct 29 | MIDTERM exam (in class) | | | | |
| 19. Mon Nov 3 | Introduction to differential equations
- DE problem statement
- GD as ODE, Reverse ODE
- SDE, Brownian / Wiener process
- Reverse SDE | - lec18a_slides_diffEq.pptx
- lec18_video | 🟢 Chan’s tutorial (Purdue), Chapter 4
| | 🖥️ MP3 |
| 20. Mon
Nov 10 | SDE, ODE, and Flows 1
- Diffusion and ODE/SDE
- Reverse ODE/SDE
- Fokker Planck and Kolmogorov
- Solvers: Euler and Runge-Kutta | - lec19a_slides_SDEDiffusion.pptx
- lec19_video
| 🟢 Flow and Diffusion tutorial (MIT), chap. 1 | | |
| 21. Wed
Nov 12 | Flows 2
- Conditional and marginal probability path
- Designing target vector field
- Continuity equation
- Zoom out → visualize
- Score functions | - lec20a_slides_flowMatc.pptx
- lec20_video | 🟢 Flow and Diffusion tutorial (MIT), chap. 2
🔵 Divergence video (Khan Academy)
| | |
| 22. Mon
Nov 17 | Flows 3: ODE to SDE to score matching
- Conditional flow matching
- Conditional to marginal
- CFM is surrogate loss for FM
- Score matching extension
- Special case: Gaussian
- For Gaussian, score comes free | - lec21a_slides_ODE-to-SDE.pptx
- lec21_video | 🟢 Flow tutorial (MIT), chapters 3 and 4
🔵 Lilian’s blog
🔵 Flows_foundations_slides.pdf (UIUC)
🟢 Flow matching paper (Meta)
🔵 Rectified flows paper (UT Austin) | | |
| 23. Wed
Nov 19 | Surgery of diffusion models
- Inpainting and outpainting
- Generalization and reproducability
- Low dimensional sub-spaces in diffusion
- Precise editing of diffusion models | - lec22a_slides_editDiffusion.pptx
- lec22_video | 🟢 RePaint paper PDF
🟢 LOCO paper PDF
🟢 ICML tutorial video by Qing Qu
| | 🚫 MP3 |
| | Thanksgiving week | | | | |
| 24. Mon
Dec 1 | Low Rank and Editing Diffusion Models
- Problem statement
- Jacobian of Denoiser
- LOCO Edit | - LOCO seminar youtube
- Answer questions in email
- Self study | 🟢 LOCO Edit paper (Michigan)
| | |
| 25. Wed
Dec 3 | NeRFs (vision and beyond)
- Radiance fields
- Rendering function and inverse problem
- Wireless NeRF | - lec24a_slides_nerf.pptx
- lec24_video | 🟢 NeRF paper (Berkeley)
🟢 Wireless NeRF paper (UIUC)
🟢 NeRFs + Diffusion paper (Berkeley) | ✍️ HW4
(Flow matching and ODEs) | |
| 26. Mon
Dec 8 | Recent papers on controlled diffusion
- RB Modulation
- Tweedie Mix
- Minority focussed T2I models | - | 🟢 MultiDiffusion paper (Weizmann)
🟢 RB_Modulation paper (UT Austin)
🟢 TweedieMix paper (KAIST)
🟢 Composing diffusion models paper (MIT)
🟢 Diffusion seeds blog (Reddit)
🔵 Progressive guidance paper (Sydney) | | |
| 27. Wed
Dec 10 | Last class:
- What else is going on in AI? | | 🔵 How LLMs work youtube (3Blue1Brown)
🔵 RL lectures 2 and 3 youtube (Berkeley) | | |
| | Conflict exam: Friday, Dec 12, 2pm, CSL 261 | Need to show proof of flight tickets bought before mid-November | | | |
| | FINAL Exam: Thursday, Dec 18, 7-10pm. | In class | | | |