Quizmaster

Learning objectives

What you will be able to explain

Why the partition function is difficult
Positive and negative phases
Energy-model gradient facts
Contrastive divergence and stochastic maximum likelihood
Short-run learning caveats
Pseudolikelihood avoids global normalization

Section 01

Why the partition function is difficult to Energy-model gradient facts

Why the partition function is difficult

Energy-based models are easy to score locally but global normalization couples all states.

Guided checkpoint

For $p(x)=\tilde p(x)/Z$ , why can maximum likelihood be intractable?

Source: Chapter 18, pp. 605-606

Positive and negative phases

The log-partition derivative becomes a model expectation, which is the computational obstacle in maximum-likelihood learning.

Guided checkpoint

Match each gradient term to its interpretation.

Source: Chapter 18, section 18.1, pp. 606-607

Energy-model gradient facts

Learning must balance fitting data against redistributing probability mass over the entire state space.

Guided checkpoint

Judge each statement.

Source: Chapter 18, section 18.1, pp. 606-607

Section 02

Contrastive divergence and stochastic maximum likelihood to Pseudolikelihood avoids global normalization

Contrastive divergence and stochastic maximum likelihood

CD is biased toward matching local reconstructions, while persistent chains aim to track the evolving model distribution.

Guided checkpoint

Match each method to its chain initialization or persistence.

Source: Chapter 18, section 18.2, pp. 607-615

Short-run learning caveats

Practical success does not make a biased estimator exact; mixing and chain initialization determine what distribution is actually contrasted with data.

Guided checkpoint

Select all correct statements.

Source: Chapter 18, section 18.2, pp. 607-615

Pseudolikelihood avoids global normalization

Each local conditional has a much smaller normalization problem, providing a consistent alternative under suitable conditions.

Guided checkpoint

What objective does it maximize?

Source: Chapter 18, section 18.3, pp. 615-617

Section 03

Pseudolikelihood tradeoffs to Denoising score matching

Pseudolikelihood tradeoffs

The method changes the objective to gain tractability; local conditional problems can still be expensive for large variable domains.

Guided checkpoint

Evaluate each claim.

Source: Chapter 18, section 18.3, pp. 615-617

Score matching and ratio matching

Both methods construct local quantities in which global normalization disappears.

Guided checkpoint

Match each idea to its defining object.

Source: Chapter 18, section 18.4, pp. 617-619

Denoising score matching

Denoising supplies a tractable regression objective connected to density-score estimation.

Guided checkpoint

What practical bridge does denoising create?

Source: Chapter 18, section 18.5, pp. 619-620

Section 04

Noise-contrastive estimation to Estimating a partition function

Noise-contrastive estimation

NCE converts unsupervised density estimation into supervised discrimination against a known noise source.

Guided checkpoint

Match each component to its role.

Source: Chapter 18, section 18.6, pp. 620-623

Class prior in NCE

There is 1 data example among

1+9=10

total examples, so the data-class prior is 0.1.

Guided checkpoint

For every data sample, NCE draws 9 noise samples. What is the prior probability that a randomly selected training example for the discriminator came from data?

Source: Chapter 18, section 18.6, pp. 620-623

Estimating a partition function

Partition estimation is itself a demanding inference problem; intermediate distributions can control otherwise catastrophic importance-weight variance.

Guided checkpoint

Match each method to its characteristic.

Source: Chapter 18, section 18.7, pp. 623-630

Section 05

Annealed importance sampling reasoning

AIS combines importance weighting with transitions along an annealing path; schedule and mixing quality strongly affect variance.

Guided checkpoint

Judge each statement.

Source: Chapter 18, section 18.7, pp. 623-630

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course