Paper Reading - [NeurIPS 2020] Denoising Diffusion Probabilistic Models

Paper reading for [NeurIPS 2020] Denoising Diffusion Probabilistic Models by Jonathan Ho, Ajay Jain and Pieter Abbeel. Paper Link is here at NeurIPS Proceedings.

好,好数学，要看不懂了

Paperlist

https://zaixiang.notion.site/Diffusion-Models-for-Deep-Generative-Learning-24ccc2e2a11e40699723b277a7ebdd64

预备知识

Gauss 分布的 KL 散度公式：

\(K L(p, q)=\log \frac{\sigma_2}{\sigma_1}+\frac{\sigma^2+\left(\mu_1-\mu_2\right)^2}{2 \sigma_2^2}-\frac{1}{2}\)

单层 VAE 原理

多层 VAE 原理与置信下界

Conclusion

• high-quality samples using diffusion models

• connections among diffusion models and:

  • variational inference for training markov chains
  • denoising score matching
  • annealed Langevin dynamics
  • energy-based models by extension
  • autoregressive models
  • progressive lossy compression

Abstract

• high-quality image synthesis using diffusion probabilistic models

• Best results: obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics