Foundations
to divergences.
From data distributions and neural generators to optimization, convexity, and comparing probability distributions.
Learning route: Generative models → data distributions → generators → training objective → optimization → convexity → f-divergence → KL / JS / total variation.
Study sections
Follow the order or choose a sectionWhat are deep generative models?
A generative model learns how to produce plausible new examples resembling data. The chapter introduces GANs, VAEs, denoising diffusion…
02The core generative-modeling problem
Given samples from an unknown P_X, our goal is to learn a mechanism that produces samples with a distribution close to P_X. The example…
03Training and sampling are different
During training, we compare real-data samples with generated samples and use an objective to update θ. During ordinary sampling after t…
04Detour D — Optimization, minimum and argmin (detailed)
An objective is a number we want to make small (or large). A minimum is the smallest attained objective VALUE. An argmin is the INPUT,…
05Detour E — Convex functions (detailed)
A function f defined on a convex domain is convex when its graph never rises above the straight chord joining two points of its graph.…
06Why convexity enters f-divergence
The example introduces variational divergence minimization through f-divergences. Under the standard density-ratio formulation, let P a…
07Examples: KL, JS and total variation
KL divergence is generated by f(u)=u ln u (with f(0)=0 by continuity), giving D_KL(P∥Q)=∫p ln(p/q). KL generally changes when P and Q a…