Quizmaster

Learning objectives

What you will be able to explain

Inference as optimization
Expectation maximization
EM guarantees and limits
MAP inference in sparse coding
Evaluate a MAP objective
Variational-inference language

Section 01

Inference as optimization to EM guarantees and limits

Inference as optimization

Approximate inference trades exact summation or integration for optimization over a restricted family.

Guided checkpoint

How can an intractable posterior $p(h|v)$ be approximated?

Source: Chapter 19, section 19.1, pp. 633-634

Expectation maximization

Exact EM uses the true posterior in the E-step; generalized or variational versions may only improve an approximation or bound.

Guided checkpoint

Match each EM object or stage to its role.

Source: Chapter 19, section 19.2, pp. 634-635

EM guarantees and limits

Monotonic local improvement is not a global-optimality guarantee, and the posterior computation may be the hard part.

Guided checkpoint

Judge each statement.

Source: Chapter 19, section 19.2, pp. 634-635

Section 02

MAP inference in sparse coding to Variational-inference language

MAP inference in sparse coding

Negative log Gaussian likelihood gives squared error and negative log Laplace prior gives an

L^1

penalty.

Guided checkpoint

For a Laplace prior on codes and Gaussian reconstruction noise, what objective does MAP inference produce?

Source: Chapter 19, section 19.3, pp. 635-638

Evaluate a MAP objective

1.2+0.25(4)=2.2

Guided checkpoint

Reconstruction cost is 1.2, $||h||_1=4$ , and the prior penalty coefficient is 0.25. Compute the total cost.

Source: Chapter 19, section 19.3, pp. 635-638

Variational-inference language

log p(v)=ELBO+D_{KL}(q(h|v)||p(h|v))

, so maximizing the bound minimizes the reverse KL gap.

Guided checkpoint

Match each object to its meaning.

Source: Chapter 19, section 19.4, pp. 638-651

Section 03

Recover the evidence from a bound to Why KL direction matters

Recover the evidence from a bound

log p(v)=ELBO+KL=-12.5+0.8=-11.7

Guided checkpoint

The ELBO is -12.5 and $D_{KL}(q(h|v)||p(h|v))=0.8$ . What is $log p(v)$ ?

Source: Chapter 19, section 19.4, pp. 638-651

Mean-field inference

Mean field gains tractability by discarding some dependencies; reverse KL often favors one mode rather than covering all modes.

Guided checkpoint

Evaluate each statement.

Source: Chapter 19, section 19.4.2, pp. 643-647

Why KL direction matters

Reverse KL strongly penalizes mass in low-density target regions, often preferring one plausible explanation when the family cannot bridge modes.

Guided checkpoint

When $q$ cannot cover two separated posterior modes, what behavior is commonly associated with minimizing $D_{KL}(q||p)$ ?

Source: Chapter 19, section 19.4, pp. 638-651

Section 04

Learned approximate inference to When approximate inference trains the model

Learned approximate inference

Learned inference exchanges per-example optimization cost for a reusable predictor, often combining speed with optional refinement.

Guided checkpoint

Match each concept to its role.

Source: Chapter 19, section 19.5, pp. 651-653

When approximate inference trains the model

In latent-variable learning, inference is inside the objective. Its errors shape which generative parameters are favored.

Guided checkpoint

Select all correct statements.

Source: Chapter 19, sections 19.4-19.5, pp. 638-653

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course