INTRODUCTION

Divided into part procession , Diffusion Model has a “Forward Process” and a “Reverse Process”.

Forward Process

Continuously adding noise into a image

Reverse Process

Continuously denoise the image that filled with noise by training a neural network

Objective

Then we could use the trained network to produce a new picture from a random noise-distribution image

VAE vs. Diffusion Model

The bottleneck part or the CODE in the VAE model is quite like the picture after adding noise N times

Training Algorithm

Take gradient descent step on

until converged

In Diffusion Model

means “sample a clean image”

means “sample a noise”,which is Target Noise

is a Noise predictor function which mix clean image and noisy images together with specific weights( and )

tips: is increasingly smaller ,which means the noise will Noise takes up a larger percentage of the image

Sampling

Algorithm

do

if else

end for

return

Maximum Likelihood Estimation

we sample {} from

So how we can compute the because the distribution is so complicated that we can’t work out the objective function easily

We want to find a that can maximum the Likelihood Estimation

So maximum likelihood estimation is to minimum the KL divergence(like VAE)

By the way, is not related to , so we could add it freely