Question 3

Suppose we can generate many forward simulations from the captcha model:

1. sample a candidate text \(x\) from the prior;
2. generate an image \(y\) from the image model given \(x\).

Now we are given a fixed observed image \(y_{\text{obs}}\).

What extra information would be needed to turn these forward simulations into information about the posterior \(p(x \given y_{\text{obs}})\)?

Vote: https://pe.app/ppls

What is the likelihood here?

For each candidate text \(x\), the model renders a blurry image \(\operatorname{render}(x)\) and compares it with the observed image \(y_{\text{obs}}\).

The likelihood is higher when the squared pixel error is smaller:

\[ y_{\text{obs}} \sim \operatorname{Normal}(\operatorname{render}(x), \sigma), \qquad \log p(y_{\text{obs}} \given x) = \text{constant} - \frac{1}{2\sigma^2} \|y_{\text{obs}}-\operatorname{render}(x)\|^2. \]

Likelihood is a soft pixel-level match between the rendered candidate and the observed degraded text.

Conditioning changes the distribution

\[p(x \given y) \;=\; \frac{p(y \given x)\,p(x)}{p(y)}\]

• Run the program forward and the latents follow the prior \(p(x)\).
• Conditioning on \(y\) reshapes them into the posterior, a different distribution: the executions that stay plausible are no longer the ones the program produces on its own.
• A query is an expectation under that posterior, and forward averages estimate the wrong one.

The lucky escape: conjugacy

In rare cases prior and likelihood are conjugate: the posterior stays in the same family and is available in closed form.
The running coin’s \(\text{Beta}(1,1)\) prior with one observed flip gives \(\text{Beta}(2,1)\), density \(2x\).

Full derivation in the Appendix. Conjugacy is the exception: table of conjugate distributions.

Samples are a usable answer

Outside conjugacy there is no formula for \(p(x \given y)\), yet samples drawn from it still answer the queries:

\[\frac{1}{L} \sum_{\ell=1}^{L} f\!\left(x^{(\ell)}\right) \;\xrightarrow{\;L \to \infty\;}\; \int f(x)\, p(x \given y)\, dx, \qquad x^{(\ell)} \sim p(x \given y)\]

• \(f = \mathbb{I}(x > 0.7)\): tail probability. \(f = x\): posterior mean. Same samples, any \(f\), chosen after sampling.
• In general we get neither an analytic posterior nor exact samples, but sampling can be approximated consistently.

Question 4

Back to Q1’s coin, but now we insist on conditioning, using plain Python:

def flip_conditioned():
    while True:
        x = random.random()              # x ~ Beta(1, 1)
        y = int(random.random() < x)     # y ~ Bernoulli(x)
        if y == 1:
            return x                     # keep only runs where y == 1

The returned values of x are distributed according to…

Vote: https://pe.app/ppls

You already wrote an inference algorithm

• That loop is rejection sampling: simulate the joint, keep runs that match the data.
• Acceptance probability given \(x\) is \(p(y{=}1 \given x)\). Kept \(x\)’s follow the posterior exactly. Histogram them against \(\text{Beta}(2,1)\): density \(2x\).
• Conditioning by if: no algebra, no integral, just the simulator plus a filter.

Question 5

Apply the same trick to Q2’s hello world, conditioning on \(y = 2.3\) (treat floats as ideal real numbers):

def gaussian_conditioned():
    while True:
        mu = random.gauss(0, 1)
        y  = random.gauss(mu, 1)
        if y == 2.3: return mu

What happens when we call it?

Vote: https://pe.app/ppls

The zero-probability wall → observe

• For continuous data, matching is hopeless: we must score runs against the data, not filter them.
• Scoring needs language support: something must mark where the data enters.

• (sample d): draw a value from distribution \(d\) and move on.
• (observe d v): declare that the value \(v\) was observed from \(d\).
• A random library makes a simulation language. observe plus an evaluator that implements conditioning makes a probabilistic programming language.

A first probabilistic program (Program 1.1)

(let [prior (beta 1 1)
      x (sample prior)
      likelihood (bernoulli x)
      y 1]
  (observe likelihood y)
  x)

• Lisp-style s-expressions: the mini-language we will build.
• The program denotes the joint and, via observe + its return value, the inference problem.

Question 6

Two programs differing in one construct:

;; Program 🚲
(let [x (sample (beta 1 1))]
  (observe (bernoulli x) 1)
  x)

;; Program 🚃
(let [x (sample (beta 1 1))
      y (sample (bernoulli x))]
  x)

How do the denoted distributions of the return value differ?

Vote: https://pe.app/ppls

What a probabilistic program means

• It simultaneously denotes:
  1. a joint \(p(x, y)\), read it as a simulator;
  2. a conditional: observe marks which variables carry data; the return value names the posterior you want.
• Nothing in the program text computes that conditional. Conditioning is what inference does, the evaluator’s job.

Denotation \(\neq\) one execution

The program denotes:

\[ p(x \given y{=}1) \]

But one execution only produces one sampled candidate for \(x\).

At the observe, the evaluator does not draw \(y\).

It must instead ask:

\[ \text{How likely is the observed value } y=1 \text{ under this sampled } x? \]

Operationally:

• sample draws values;
• observe draws nothing;
• observe updates the score.

Density, not probability

For the Gaussian model, rejection tried to draw a fresh \(y'\) and test:

\[ y' = 2.3 \]

That event has probability \(0\), so the loop almost surely never returns.

observe asks a different question. For a sampled \(\mu\), it evaluates the density of the fixed observed value:

\[ w \;=\; p(2.3 \given \mu) \;=\; \frac{1}{\sqrt{2\pi}} e^{-(2.3-\mu)^2/2}. \]

• This is not the probability of the point \(2.3\).
• It is a score: how well this sampled \(\mu\) explains the observation.
• Every run returns; runs with better explanations receive larger weights.

The question of this course

We can describe the evaluator in words. How do you implement such a language, inside Python?

Same observe, three embeddings: the spine of the course.

Summary

Questions: Answer Key (1/2)

Questions: Answer Key (2/2)

Conjugacy: the Beta-Bernoulli model

The coin’s prior is a Beta density on \(x \in [0,1]\), normalized by the Beta function \(B\):

\[ \begin{aligned} \text{Beta}(x;\, a, b) &= \frac{x^{a-1}(1-x)^{b-1}}{B(a,b)}, \\ B(a,b) &= \int_0^1 x^{a-1}(1-x)^{b-1}\,dx = \frac{\Gamma(a)\,\Gamma(b)}{\Gamma(a+b)}. \end{aligned} \]

With prior \(\text{Beta}(x;\alpha,\beta)\) and one observed flip, algebra returns a posterior in the same family:

\[p(x \given y) \;=\; \text{Beta}(x;\; \alpha + y,\; \beta - y + 1)\]

• At integer arguments \(\Gamma(n) = (n-1)!\), so the constant is exact and easy.
• Uniform prior \(\text{Beta}(x;1,1) = 1\), observe heads: posterior \(\text{Beta}(x;2,1)\), density \(2x\). Prior and likelihood in a known family: conjugacy.

Deriving \(p(x \given y)\)

Prior \(p(x) = x^{\alpha-1}(1-x)^{\beta-1}/B(\alpha,\beta)\) and one Bernoulli observation \(p(y \given x) = x^{y}(1-x)^{1-y}\). Their product collects the exponents:

\[p(x,y) \;=\; p(x)\,p(y \given x) \;=\; \frac{x^{(\alpha+y)-1}\,(1-x)^{(\beta-y+1)-1}}{B(\alpha,\beta)}\]

Integrate over \(x\) for the evidence; the integral is again a Beta function:

\[p(y) \;=\; \int_0^1 p(x,y)\,dx \;=\; \frac{B(\alpha+y,\; \beta-y+1)}{B(\alpha,\beta)}\]

Divide; the constant \(B(\alpha,\beta)\) cancels and a normalized Beta density remains:

\[p(x \given y) \;=\; \frac{p(x,y)}{p(y)} \;=\; \frac{x^{(\alpha+y)-1}\,(1-x)^{(\beta-y+1)-1}}{B(\alpha+y,\; \beta-y+1)} \;=\; \text{Beta}(x;\; \alpha+y,\; \beta-y+1)\]