Yu-Lin Wei (UIUC)
Romit Roy Choudhury (UIUC)
See more prerequisite material below
Slides from the tutorial here:
1_Probability_warmup_Bayes_MLE_EM.pptx
4_Diffusion2_score_langevin.pptx
All slides in a single deck:
2024_genModel_tutorial_nov10.pptx
Videos from the tutorial here:
Module 0/6: Few words before we start
Module 1/6: Whole tutorial in a nutshell
Module 2/6: Math foundations (Bayes, MLE, EM)
Module 3/6: Autoencoders (AE) and Variational AE
Module 4/6: Diffusion Models
Module 5/6: Diffusion Models (Langevin and Score)
Module 6/6: GANs and Normalizing Flows
Full playlist link: here
| Prerequisites for the tutorial | Sub-topics | Compact material | |
|---|---|---|---|
| Probability | Conditional probability, Bayes’ rule, conditional independence. | 🟢 Pishro-Nik’s notes | |
| Chapter 1.4 | |||
| Random variables (discrete and continuous), distributions, independence, functions of RVs. | 🟢 Pishro-Nik’s notes | ||
| Chapters 3 and 4 | |||
| Distributions (especially exp. and Gaussian), joint distributions, covariance, bivariate Normal. | 🟢 Pishro-Nik’s notes | ||
| Chapters 4.2, 5.1, 5.3 | |||
| Random vectors, bounds and inequalities. | 🟢 Pishro-Nik’s notes | ||
| Chapters 6.1.5, 6.2.4, 6.2.5, | |||
| Statistical inference I: random sampling, point estimation, Max Likelihood (MLE). | 🟢 Pishro-Nik’s notes | ||
| Chapters 8.1.1, 8.2 | |||
| Statistical inference II: Bayesian inference, MAP, comparison to MLE, estimation of random vectors. | 🟢 Pishro-Nik’s notes | ||
| Chapters 9.1.0 — 9.1.3, 9.1.7 | |||
| Sampling from distributions. | 🟢 Just first 10 pages: MCMC sampling PDF | ||
| Linear algebra | Vector space, null space, linear independence, rank, basis, norms, least square. | 🟢 Roy Choudhury’s course | |
| Video lectures 2, 3, 4, 5 | |||
| Singular value decomposition (SVD). | 🟢 Roy Choudhury’s SVD lecture | ||
| Optimization | Hessian, saddle points, gradient descent. | 🟢 Roy Choudhury’s lecture part (a) and part (b) |
| Main tutorial (Nov 18) | Topics | Links (intuitive) | Links (mathematical) |
|---|---|---|---|
| 8:45 - 9:45am | Whole Tutorial in a Nutshell and some Recap | ||
| Overview of the 4 Pillars of Generative Models | |||
| Probability Foundations Recap | |||
| Expectation Maximization (EM) algorithm | 🟢 Roy Choudhury EM lecture and notes | 🔴 Tutorial PDF | |
| Singular Value Decomposition (SVD) recap | |||
| 10:00 - 11:30am | AutoEncoder, Variational Inference, and VAE | ||
| AutoEncoder (AE) to Variational Inference to VAE | 🟢 Abbeel variational inference lecture | 🔴 Google book | |
| Visualization of ELBO (Evidence Lower Bound) | |||
| Posterior collapse, Vector Quantized (VQ-VAE) | 🟢 Discrete representation learning paper | 🔴 NeurIPS paper | |
| Applications and Limitations | 🟢 Google empirical paper | ||
| 🟢 AlterNet paper | 🔴 Prior matching paper | ||
| 11:45 - 12:45pm | From Hierarchical VAE to Diffusion | ||
| Hierarchical VAE and Markov Process | 🟢 Luo’s tutorial | 🔴 Google tutorial PDF | |
| Variational Diffusion Models (VDM) #1 | 🔴 Stanley Chan tutorial | ||
| 12:50 - 2:00pm | Lunch | ||
| 2:00 - 2:45pm | From Hierarchical VAE to Diffusion (contd.) | ||
| Variational Diffusion Models (VDM) #2 | 🟢 Latent Diffusion | ||
| Analogy and Visualization | |||
| DDPM (and why Gaussians make it elegant) | 🟢 DDPM paper | ||
| Applications | |||
| 3:00 - 3:30pm | Another Way to Understand Diffusion --> Score-based Sampling | ||
| Score function and Denoising | |||
| 🟢 Yang Song’s blog | |||
| Basic idea of Langevin and Annealing | 🟢 Michael Elad’s lecture | 🔴 Score matching proof PDF | |
| DDPM and More Intuition | |||
| 3:45 - 4:45pm | GANs and flows | ||
| Implicit Density Models | |||
| Adversarial Training and GAN | 🟢 Adversarial autoencoder paper | 🔴 NeurIPS tutorial | |
| Problems - Data, Mode Collapse, Stability | |||
| Invertible Transformations and Change of Variables | |||
| Normalizing Flows | 🟢 Lilian’s blog | 🔴 RealNVP paper | |
| Estimating Probability | |||
| 5:00 - 5:30pm | Wrapping Up | ||
| Connecting all the Dots for the Bigger Picture | |||
| Questions, Applications, and Discussion |