Change of notation

Today we will denote the latent variables by \(z\) and the observed data by \(x\). A probabilistic program defines a joint over latents and data: \[ p_\theta(x,z) = p(z)\,p_\theta(x \given z). \]

• We observe \(x\) and want the posterior \(p_\theta(z \given x)\).
• Inference must generate latent executions \(z\) that could have produced \(x\).

Question 1

A model gives a joint \(p_\theta(x,z)=p(z)\,p_\theta(x \given z)\), and we introduce a guide \(q_\phi(z \given x)\).

What role does the guide play?

A guide proposes executions

The model defines the target.

The guide defines how candidate latent executions are proposed:

\[z \sim q_\phi(z \given x).\]

The guide does not replace the model. It is part of the inference mechanism.

Question 2

Likelihood weighting samples latent variables from the prior, \(z \sim p(z),\) and scores the observation. Now suppose we instead sample

\[z \sim q_\phi(z \given x).\]

What has changed relative to likelihood weighting?

Proposal, not target

• The guide changes how candidate executions are generated.
• The model still defines which executions are plausible.
• The correction is still with respect to the model joint \(p_\theta(x,z)\), whose normalized version is the posterior \(p_\theta(z \given x)\).
• Conditioning the proposal on \(x\) can reduce weight variance, but does not remove the correction in general.

Question 3

For importance sampling with proposal \(q(z \given x)\) and weight \(w(z)=\dfrac{p_\theta(x,z)}{q(z \given x)}\),

which proposal would make the weights constant?

The zero-variance proposal

The ideal proposal is the posterior:

\[q^\star(z \given x)=p_\theta(z \given x).\]

Then

\[w(z) = \frac{p_\theta(x,z)}{p_\theta(z \given x)} = p_\theta(x),\]

which does not depend on \(z\).

This is ideal but unavailable, because sampling from \(p_\theta(z \given x)\) is the inference problem itself.

Question 4

The two KL directions are

\[\mathrm{KL}(q_\phi \Given p_\theta) = \mathbb{E}_{q_\phi(z \given x)}\!\left[\log q_\phi(z \given x) - \log p_\theta(z \given x)\right],\]

and

\[\mathrm{KL}(p_\theta \Given q_\phi) = \mathbb{E}_{p_\theta(z \given x)}\!\left[\log p_\theta(z \given x) - \log q_\phi(z \given x)\right].\]

The first is often more mode-seeking; the second is often more mode-covering.

Why does ordinary VI naturally lead to the first direction?

What we can actually compute

Ordinary VI uses expectations under the guide:

\[z \sim q_\phi(z \given x).\]

That is operationally available because we control \(q_\phi\).

The inclusive direction would require expectations under

\[p_\theta(z \given x),\]

which is exactly the posterior we are trying to approximate.

Question 5

For fixed \(\theta\), suppose we want to fit \(q_\phi(z \given x)\) by minimizing \(\mathbb{E}_{q_\phi(z \given x)}\!\left[\log q_\phi(z \given x) - \log p_\theta(z \given x)\right].\)

But, the ELBO is: \[ \mathbb{E}_{q_\phi(z \given x)}\!\left[\log p_\theta(x,z) - \log q_\phi(z \given x)\right].\]

Why do we usually optimize the ELBO instead of this KL directly?

From the exclusive KL to the ELBO

Start from

\[\mathrm{KL}(q_\phi \Given p_\theta) = \mathbb{E}_{q_\phi}\!\left[\log q_\phi(z \given x) - \log p_\theta(x,z)\right] + \log p_\theta(x).\]

Rearrange:

\[\log p_\theta(x) = \mathcal{L}(\theta,\phi) + \mathrm{KL}(q_\phi \Given p_\theta),\]

where

\[\mathcal{L}(\theta,\phi) = \mathbb{E}_{q_\phi(z \given x)}\!\left[\log p_\theta(x,z) - \log q_\phi(z \given x)\right].\]

For fixed \(\theta\), \(\log p_\theta(x)\) does not depend on \(\phi\), so maximizing \(\mathcal{L}\) minimizes the exclusive KL.

Question 6

The ELBO is

\[\mathcal{L}(\theta,\phi) = \mathbb{E}_{q_\phi(z \given x)}\!\left[\log p_\theta(x,z) - \log q_\phi(z \given x)\right].\]

We want to estimate \(\nabla_\phi \mathcal{L}(\theta,\phi)\).

What is the main difficulty?

The gradient problem

• Gradients with respect to \(\theta\) are comparatively direct once \(z\) is sampled.
• Gradients with respect to \(\phi\) are subtler because \(\phi\) changes the law of the sampled execution.
• Two standard estimators are score-function gradients and reparameterized gradients.

Two gradient estimators

Reparameterization is the standard route for Gaussian latent VAEs.

Question 7

Suppose we know part of the data-generating structure and write \(z \sim p(z)\), but the relation between latent structure and observations is complex. We therefore write \[ x \sim p_\theta(x \given z), \]

where \(p_\theta(x \given z)\) is represented by a neural network.

What is gained by making this part of the model deep?

Partial specification

\[p_\theta(x,z)=p(z)\,p_\theta(x \given z).\]

Deep probabilistic programming does not mean replacing the probabilistic model with a neural network.

It means:

• specify the structure we know;
• learn the parts we do not want to hand-design;
• keep the model generative and probabilistic.

Question 8

Suppose a model uses

\[ \eta_\theta(z)=f_\theta(z), \qquad x \sim p(x;\eta_\theta(z)), \]

where \(p(x;\eta)\) is a known distribution family.

What is the main reason for this pattern?

Why output distribution parameters?

A probabilistic program needs distributions with operations like:

\[ x \sim p_\theta(x \given z) \]

and

\[ \log p_\theta(x \given z). \]

A neural network can make the distribution flexible by predicting parameters,

\[ \eta_\theta(z) = f_\theta(z), \qquad p_\theta(x \given z)=p(x;\eta_\theta(z)), \]

while the chosen family \(p(x;\eta)\) supplies:

• a sampler;
• a density or log-density;
• gradients with respect to \(\eta\) and \(\theta\), when needed.

So the network does not usually replace the probability distribution. It parameterizes one.

Normalizing flows

We are not limited to very simple families.

Start with a base distribution that we know how to sample and score:

\[ u \sim p_0(u). \]

Apply an invertible neural transformation: \(x = T_\theta(u).\)

Then samples are easy:

\[ u \sim p_0(u), \qquad x = T_\theta(u). \]

And densities are available by change of variables:

\[ \log p_\theta(x) = \log p_0(T_\theta^{-1}(x)) + \log \Bigl| \det \frac{\partial T_\theta^{-1}(x)}{\partial x} \Bigr|. \]

A normalizing flow is a more expressive distribution family supporting the two needed operations:

• sampling;
• log-density evaluation.

Amortized inference

Suppose that for each observation \(x\) we want to infer the latent \(z\).

\[ p_\theta(z \given x). \]

A guide shares parameters across observations:

\[ \lambda_\phi(x) = \text{network}_\phi(x), \qquad q_\phi(z \given x) = q(z;\lambda_\phi(x)). \]

Instead of solving a new inference problem from scratch for every \(x\), we learn a function that maps observations to proposal distributions over latents.

Question 9

A variational autoencoder uses

\[z \sim p(z), \qquad x \sim p_\theta(x \given z), \qquad z \sim q_\phi(z \given x).\]

The terms \(p_\theta(x \given z)\) and \(q_\phi(z \given x)\) are often called the decoder and encoder.

How should we interpret these two networks in probabilistic-programming terms?

The VAE, assembled

Model:

\[p_\theta(x,z)=p(z)\,p_\theta(x \given z).\]

Guide:

\[q_\phi(z \given x).\]

Objective:

\[\mathcal{L}(\theta,\phi) = \mathbb{E}_{q_\phi(z \given x)}\!\left[\log p_\theta(x,z) - \log q_\phi(z \given x)\right].\]

Common Gaussian guide:

\[q_\phi(z \given x) = \mathcal{N}\!\left(\mu_\phi(x),\,\operatorname{diag}(\sigma_\phi^2(x))\right),\]

with reparameterization

\[z = \mu_\phi(x) + \sigma_\phi(x)\odot\epsilon, \qquad \epsilon \sim \mathcal{N}(0,I).\]

Summary

• A guide is a learned proposal over latent executions.
• The ideal proposal is the posterior, but it is not directly available.
• VI learns a tractable guide \(q_\phi(z \given x)\).
• The available KL direction uses expectations under \(q_\phi\).
• The ELBO is the tractable objective used to train the guide.
• Gradients through stochastic choices require score-function or reparameterized estimators.
• Learning \(\theta\) trains the generative model, often partially specified with neural conditionals.
• A VAE is a neural generative model plus an amortized neural guide.

Questions: Answer Key (1/2)

Questions: Answer Key (1/2)