← Mathematics for Machine Learning

Learn / Mathematics for Machine Learning

10 - Dimensionality Reduction with Principal Component Analysis

Advanced mastery of Chapter 10, covering every named section and exercise-style synthesis across book pages 317-343.

Learning path

0%

0 of 4 sections marked complete · about 42 minutes

Learning objectives

What you will be able to explain

  • PCA problem setting
  • Maximum-variance direction
  • PCA spectrum
  • Projection perspective
  • Two PCA perspectives
  • Truncated reconstruction error

Section 01

PCA problem setting to PCA spectrum

01

PCA problem setting

PCA is an unsupervised linear subspace method.

Guided checkpoint

For centered data.

Source: Sec. 10.1, pp. 318-320

02

Maximum-variance direction

The Rayleigh quotient is maximized by the leading covariance eigenvector.

Guided checkpoint

The first principal direction solves:

Source: Sec. 10.2, pp. 320-325

03

PCA spectrum

Covariance is positive semidefinite.

Guided checkpoint

For covariance SS.

Source: Sec. 10.2, pp. 320-325

Section 02

Projection perspective to Truncated reconstruction error

01

Projection perspective

UkUkTU_kU_k^T is the orthogonal projector onto the principal subspace.

Guided checkpoint

Which reconstruction uses orthonormal basis UkU_k?

Source: Sec. 10.3, pp. 325-333

02

Two PCA perspectives

Total variance decomposes into retained plus discarded orthogonal variance.

Guided checkpoint

Judge the equivalence.

Source: Secs. 10.2-10.3, pp. 320-333

03

Truncated reconstruction error

Only the smallest eigen-direction is discarded.

Guided checkpoint

Covariance eigenvalues are 9,4,1. What variance is discarded by rank-2 PCA?

Source: Sec. 10.3, pp. 325-333

Section 03

Eigenvector computation and low rank to PCA in practice

01

Eigenvector computation and low rank

SVD avoids forming covariance and exposes the same principal subspace.

Guided checkpoint

Match method.

Source: Sec. 10.4, pp. 333-335

02

PCA in high dimensions

Nonzero spectra of centered-data products are linked.

Guided checkpoint

When features greatly outnumber samples, what can reduce eigenproblem size?

Source: Sec. 10.5, pp. 335-336

03

PCA in practice

Center, decompose, select, and project form the practical workflow.

Guided checkpoint

Order the conceptual steps by matching role.

Source: Sec. 10.6, pp. 336-339

Section 04

Latent-variable perspective to Exercise-style explained variance

01

Latent-variable perspective

PPCA turns geometric compression into a generative model.

Guided checkpoint

For probabilistic PCA.

Source: Sec. 10.7, pp. 339-343

02

Further-reading bridge

Kernel methods replace explicit coordinates by inner products in feature space.

Guided checkpoint

Which family generalizes PCA to nonlinear embeddings using kernels?

Source: Sec. 10.8, p. 343

03

Exercise-style explained variance

The top two explain (6+3)/(6+3+1)=.9(6+3)/(6+3+1)=.9.

Guided checkpoint

Eigenvalues are 6,3,1. What fraction of variance is explained by two PCs?

Source: Chapter 10 synthesis, pp. 317-343

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course