Quizmaster

Learning objectives

What you will be able to explain

Boltzmann-machine probability
General Boltzmann machines
The restriction in an RBM
RBM hidden activation
RBM operations
Deep belief network structure

Section 01

Boltzmann-machine probability to The restriction in an RBM

Boltzmann-machine probability

Lower-energy configurations receive larger unnormalized probability, and the partition function normalizes over all visible and hidden states.

Guided checkpoint

How does a Boltzmann machine convert an energy $E(v,h)$ into probability?

Source: Chapter 20, section 20.1, pp. 654-656

General Boltzmann machines

General connectivity creates expressive dependencies but also couples hidden inference and global normalization.

Guided checkpoint

Judge each statement.

Source: Chapter 20, section 20.1, pp. 654-656

The restriction in an RBM

The bipartite graph makes hidden units conditionally independent given visibles and visibles conditionally independent given hiddens.

Guided checkpoint

Which edges are prohibited?

Source: Chapter 20, section 20.2, pp. 656-660

Section 02

RBM hidden activation to Deep belief network structure

RBM hidden activation

\sigma(0)=1/(1+e^0)=0.5

Guided checkpoint

For a binary RBM hidden unit, $p(h_j=1|v)=\sigma(b_j+W_j^Tv)$ . If the pre-sigmoid value is 0, what is the probability?

Source: Chapter 20, section 20.2, pp. 656-660

RBM operations

RBM structure makes conditional inference easy even though marginal likelihood and equilibrium sampling remain challenging.

Guided checkpoint

Match each operation to the property that enables it.

Source: Chapter 20, section 20.2, pp. 656-660

Deep belief network structure

A DBN is a hybrid directed-undirected generative model, distinct from a deep Boltzmann machine.

Guided checkpoint

Match each part of a DBN to its role.

Source: Chapter 20, section 20.3, pp. 660-663

Section 03

Deep Boltzmann machine distinction to Boltzmann machines for real-valued data

Deep Boltzmann machine distinction

DBMs distribute posterior explanation across layers but require approximate inference for hidden states.

Guided checkpoint

What defines a DBM?

Source: Chapter 20, section 20.4, pp. 663-676

DBM inference and training

Unlike an RBM, explaining-away interactions across multiple hidden layers make exact posterior inference intractable.

Guided checkpoint

Evaluate each claim.

Source: Chapter 20, section 20.4, pp. 663-676

Boltzmann machines for real-valued data

Moving beyond binary data requires valid conditional distributions and careful control of variance and interaction scale.

Guided checkpoint

Match each unit or transformation to its purpose.

Source: Chapter 20, section 20.5, pp. 676-683

Section 04

Convolutional Boltzmann machine to Extensions of Boltzmann machines

Convolutional Boltzmann machine

Convolutional sharing lets the energy model detect the same latent feature at multiple positions efficiently.

Guided checkpoint

Which inductive bias does it add?

Source: Chapter 20, section 20.6, pp. 683-685

Boltzmann machines for structured outputs

Conditional energy models can represent dependencies among outputs while conditioning on complex observed inputs.

Guided checkpoint

Match each component to its function.

Source: Chapter 20, section 20.7, pp. 685-686

Extensions of Boltzmann machines

The energy-based framework supports many unit types, structures, and conditioning patterns.

Guided checkpoint

Which dimensions can be varied in the broader family?

Source: Chapter 20, section 20.8, pp. 686-687

Section 05

Back-propagation through random operations to Directed generative networks

Back-propagation through random operations

Both estimators are unbiased under suitable conditions, but pathwise derivatives often have lower variance when a differentiable reparameterization exists.

Guided checkpoint

Match each estimator to its key idea.

Source: Chapter 20, section 20.9, pp. 687-692

Gaussian reparameterization

z=2+3(-0.5)=0.5

Guided checkpoint

Let $z=\mu+\sigma\epsilon$ with $\mu=2$ , $\sigma=3$ , and sampled $\epsilon=-0.5$ . Compute $z$ .

Source: Chapter 20, section 20.9, pp. 687-692

Directed generative networks

Directed models avoid a global partition function but may face posterior inference or autoregressive sampling costs.

Guided checkpoint

Match each model idea to its factorization or training device.

Source: Chapter 20, section 20.10, pp. 692-711

Section 06

Variational autoencoder reasoning to Generative stochastic networks

Variational autoencoder reasoning

Reparameterization enables low-variance gradients through continuous latent samples while the ELBO handles intractable evidence.

Guided checkpoint

Judge each statement.

Source: Chapter 20, section 20.10.3, pp. 700-711

Drawing samples from an autoencoder

Generative sampling needs a model of where valid codes lie or a transition process whose stationary distribution matches the data.

Guided checkpoint

Why can an ordinary deterministic autoencoder not automatically generate valid novel samples by drawing arbitrary codes?

Source: Chapter 20, section 20.11, pp. 711-714

Generative stochastic networks

GSNs define generation through a learned transition operator rather than requiring an explicit tractable density.

Guided checkpoint

Match each element to its role.

Source: Chapter 20, section 20.12, pp. 714-716

Section 07

Other generation schemes to Generative-evaluation caveats

Other generation schemes

Generative procedures differ in whether they expose likelihood, use latent variables, or require iterative sampling.

Guided checkpoint

Select all general routes to generation discussed around this section.

Source: Chapter 20, section 20.13, pp. 716-717

Evaluating generative models

No single metric captures fidelity, diversity, likelihood, and usefulness. Visual quality alone can hide memorization or mode dropping.

Guided checkpoint

Match each criterion to what it probes.

Source: Chapter 20, section 20.14, pp. 717-720

Generative-evaluation caveats

The conclusion emphasizes unresolved tradeoffs among tractable learning, inference, sampling, and evaluation.

Guided checkpoint

Evaluate each statement.

Source: Chapter 20, sections 20.14-20.15, pp. 717-720

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course