← Mathematics for Machine Learning

Learn / Mathematics for Machine Learning

11 - Density Estimation with Gaussian Mixture Models

Advanced mastery of Chapter 11, covering every named section and exercise-style synthesis across book pages 348-368.

Learning path

0%

0 of 4 sections marked complete · about 42 minutes

Learning objectives

What you will be able to explain

  • GMM density
  • GMM structure
  • GMM parameters
  • Mixture maximum likelihood
  • Responsibilities
  • Responsibility calculation

Section 01

GMM density to GMM parameters

01

GMM density

Mixture weights are nonnegative and sum to one.

Guided checkpoint

A KK-component GMM density is:

Source: Sec. 11.1, pp. 349-350

02

GMM structure

Label permutations represent the same mixture density.

Guided checkpoint

Judge each claim.

Source: Sec. 11.1, pp. 349-350

03

GMM parameters

Each component has probability, location, and shape parameters.

Guided checkpoint

Match parameter to constraint.

Source: Sec. 11.1, pp. 349-350

Section 02

Mixture maximum likelihood to Responsibility calculation

01

Mixture maximum likelihood

Latent assignments and log-sum coupling motivate EM.

Guided checkpoint

Why is direct optimization difficult?

Source: Sec. 11.2, pp. 350-360

02

Responsibilities

Bayes combines mixture weight and component density, normalized across components.

Guided checkpoint

The responsibility gammankgamma_{nk} is:

Source: Sec. 11.2, pp. 350-360

03

Responsibility calculation

Normalize .5(.8).5(.8) by .5(.2)+.5(.8).5(.2)+.5(.8).

Guided checkpoint

Two components have equal weights and densities at xx of .2 and .8. Compute responsibility of component 2.

Source: Sec. 11.2, pp. 350-360

Section 03

EM steps to EM properties

01

EM steps

EM alternates inference of latent variables and weighted parameter estimation.

Guided checkpoint

Match phase to operation.

Source: Sec. 11.3, pp. 360-363

02

GMM M-step updates

M-step formulas are soft-count analogues of Gaussian estimates.

Guided checkpoint

Let Nk=sumngammankN_k=sum_n gamma_{nk}.

Source: Sec. 11.3, pp. 360-363

03

EM properties

Monotonicity is not global optimality; regularization and restarts help.

Guided checkpoint

Judge each claim.

Source: Sec. 11.3, pp. 360-363

Section 04

Latent-variable perspective to Exercise-style mixture mean

01

Latent-variable perspective

The latent-variable factorization makes EM derivations transparent.

Guided checkpoint

Match probability.

Source: Sec. 11.4, pp. 363-368

02

Further-reading bridge

Variational methods optimize tractable approximations when exact E-steps fail.

Guided checkpoint

Which broader framework generalizes EM to intractable latent posteriors?

Source: Sec. 11.5, p. 368

03

Exercise-style mixture mean

Mixture mean is .4(−1)+.6(4)=2.4(-1)+.6(4)=2.

Guided checkpoint

A scalar GMM has weights .4,.6 and means -1,4. Compute its mean.

Source: Chapter 11 synthesis, pp. 348-368

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course