Tag: elbo

All the articles with the tag "elbo".

• VAEs and GANs: Two Ways to Sample What You Can't Write Down

  Updated:19 Oct, 2026

  A language model samples by writing the distribution down one token at a time. For images that means hundreds of thousands of sequential steps in an arbitrary pixel order, so two other answers grew up: learn an encoder and maximize a bound on the likelihood (the VAE), or learn a critic and fool it (the GAN). The ELBO derived and checked exactly on a model small enough to solve, the reparameterization trick, the KL term worked by hand, the GAN's optimal discriminator and its Jensen-Shannon value checked numerically, and all three models (VAE, GAN, WGAN-GP) trained on the same eight-mode test, where off-mode samples, a dropped mode and an outright training failure all show up, with a test of where the VAE's off-mode samples come from.