Quizmaster

Learning objectives

What you will be able to explain

What does PCA optimize?
Center a tiny dataset
Compute a covariance matrix
What is the first principal direction here?
Compute an explained-variance ratio
Project onto a one-dimensional principal direction

Section 01

What does PCA optimize? to Compute a covariance matrix

What does PCA optimize?

PCA searches for orthogonal directions that capture variance efficiently. The first principal component is the variance-maximizing direction.

Guided checkpoint

Choose the best description of the first principal component.

Center a tiny dataset

The mean is

((1+3)/2, (1+3)/2) = (2, 2)

. Subtracting it from the first point gives

(-1, -1)

. PCA is usually performed after centering.

Guided checkpoint

Consider the two points $(1, 1)$ and $(3, 3)$ . Compute the mean vector and the centered version of the first point.

Compute a covariance matrix

Here

X^\top X = \begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix}

, and dividing by

n = 2

gives

\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}

. The equal off-diagonal entries indicate perfect positive correlation.

Guided checkpoint

Using the centered points $(-1, -1)$ and $(1, 1)$ , compute the population covariance matrix $\frac{1}{n}X^\top X$ .

Section 02

What is the first principal direction here? to Project onto a one-dimensional principal direction

What is the first principal direction here?

All variance lies along the line

y = x

, so the first principal direction is along

(1, 1)

. The orthogonal direction

(1, -1)

has zero variance here.

Guided checkpoint

For covariance matrix $\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}$ , choose a direction of the first principal component.

Compute an explained-variance ratio

The total variance is

5 + 2 + 1 = 8

. The first component explains

5/8 = 0.625

of that total.

Guided checkpoint

If the covariance eigenvalues are $5$ , $2$ , and $1$ , what fraction of total variance is explained by the first principal component?

Project onto a one-dimensional principal direction

The coefficient is

2/\sqrt{2} = \sqrt{2}

. Multiplying that by

u

gives

(1, 1)

, which is the closest point to

(2, 0)

on the subspace spanned by

u

Guided checkpoint

Let $u = \left(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}\right)$ and $x = (2, 0)$ . Compute the projection coefficient $u^\top x$ and the reconstructed point $(u^\top x)u$ .

Section 03

PCA facts: true or false to How is PCA related to SVD?

PCA facts: true or false

PCA is unsupervised and geometry-driven. Orthogonality and covariance eigenvectors are core facts, and scaling matters because variance is scale dependent.

Guided checkpoint

Mark each statement as true or false.

Match each PCA term to its meaning

These terms show up often in PCA discussions: loadings define directions, scores are projected coordinates, and explained variance quantifies compression quality.

Guided checkpoint

Match the term to the best description.

How is PCA related to SVD?

For centered data, the right singular vectors of the data matrix correspond to eigenvectors of

X^\top X

, which are the principal directions.

Guided checkpoint

Choose the best statement for centered data.

Section 04

What does whitening do after PCA? to What can happen if one feature scale dominates and the data is not standardized?

What does whitening do after PCA?

Whitening uses the PCA basis and rescales by eigenvalues so that transformed features are decorrelated and standardized in variance.

Guided checkpoint

Choose the best answer.

What can happen if one feature scale dominates and the data is not standardized?

Because PCA follows variance, raw scale can dominate the result. Standardization is often important when features are measured in incomparable units.

Guided checkpoint

Choose the most likely consequence.

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course