Deep Learning Book (Goodfellow, Bengio & Courville)

Learn / Deep Learning Book

02 - Linear Algebra

Advanced mastery of Chapter 2, Linear Algebra, covering every named section across book pages 31-52.

Learning path

0%

0 of 5 sections marked complete · about 46 minutes

Learning objectives

What you will be able to explain

  • Scalars, vectors, matrices, and tensors
  • Compute one matrix-product entry
  • Matrix and element-wise products
  • When does the inverse solve a linear system?
  • Span, dependence, and solvability
  • Choose the norm or quantity

Section 01

Scalars, vectors, matrices, and tensors to Matrix and element-wise products

01

Scalars, vectors, matrices, and tensors

The distinction is by array dimensionality and indexing. The chapter also establishes row, column, transpose, broadcasting, and element-wise notation used later.

Guided checkpoint

Match each object to the most precise description.

Source: Chapter 2, section 2.1, pp. 31-34

02

Compute one matrix-product entry

The second row of AA dotted with the second column of BB is 0(1)+(1)(0)+4(3)=120(1)+(-1)(0)+4(3)=12.

Guided checkpoint

Let A=[[1,2,3],[0,1,4]]A=[[1,2,3],[0,-1,4]] and B=[[2,1],[1,0],[1,3]]B=[[2,1],[1,0],[-1,3]]. Compute (AB)2,2(AB)_{2,2}.

Source: Chapter 2, section 2.2, pp. 34-36

03

Matrix and element-wise products

Order reverses under transposition; matrix products generally do not commute, while the Hadamard product is entrywise.

Guided checkpoint

Judge each statement for compatible real matrices.

Source: Chapter 2, sections 2.1-2.2, pp. 33-36

Section 02

When does the inverse solve a linear system? to Choose the norm or quantity

01

When does the inverse solve a linear system?

A two-sided inverse exists for a nonsingular square matrix and yields the unique solution. Singular or rectangular systems require a more careful existence and uniqueness analysis.

Guided checkpoint

For Ax=bAx=b, which statement is the most precise?

Source: Chapter 2, section 2.3, pp. 36-37

02

Span, dependence, and solvability

Column span controls existence, while null-space freedom controls uniqueness. Dependence means at least one vector is a combination of the others.

Guided checkpoint

Select all correct statements.

Source: Chapter 2, section 2.4, pp. 37-39

03

Choose the norm or quantity

These norms measure size in different ways. Squared L2L^2 is convenient analytically but changes slowly near zero; L1L^1 is often associated with sparsity.

Guided checkpoint

Match each expression or property to its name.

Source: Chapter 2, section 2.5, pp. 39-40

Section 03

Special matrices and vectors to What eigendecomposition tells us

01

Special matrices and vectors

Normalization fixes length, not pairwise angle. An orthonormal set combines unit norms with mutual orthogonality, and an orthogonal matrix has orthonormal rows and columns.

Guided checkpoint

Mark each statement as true or false.

Source: Chapter 2, section 2.6, pp. 40-42

02

Infer an eigenvalue

Av=[3,0]T=3vAv=[3,0]^T=3v, so vv is an eigenvector with eigenvalue 3.

Guided checkpoint

For A=[[3,1],[0,2]]A=[[3,1],[0,2]] and v=[1,0]Tv=[1,0]^T, find the eigenvalue λ\lambda satisfying Av=λvAv=\lambda v.

Source: Chapter 2, section 2.7, pp. 42-44

03

What eigendecomposition tells us

Symmetric matrices have especially well-behaved spectra. General matrices may be defective or require complex eigenvectors.

Guided checkpoint

Evaluate each claim.

Source: Chapter 2, section 2.7, pp. 42-44

Section 04

SVD and the pseudoinverse to Correct the trace computation

01

SVD and the pseudoinverse

SVD exists for every real matrix. The Moore-Penrose pseudoinverse reverses the nonzero singular-value scalings and supports least-squares or minimum-norm solutions.

Guided checkpoint

Match each role to the relevant SVD component or derived object for A=UDVTA=UDV^T.

Source: Chapter 2, sections 2.8-2.9, pp. 44-46

02

Use trace cyclicity

AB=[[4,1],[8,1]]AB=[[4,-1],[8,-1]], so the trace is 4+(1)=34+(-1)=3. Rechecking directly gives: row 1 dot column 1 is 4 and row 2 dot column 2 is 34=13-4=-1, hence 3.

Guided checkpoint

Let A=[[1,2],[3,4]]A=[[1,2],[3,4]] and B=[[0,1],[2,1]]B=[[0,1],[2,-1]]. Compute Tr(AB)\operatorname{Tr}(AB).

Source: Chapter 2, section 2.10, pp. 46-47

03

Correct the trace computation

The diagonal of ABAB is (4,1)(4,-1), so Tr(AB)=3\operatorname{Tr}(AB)=3. This question deliberately catches arithmetic drift in the preceding derivation.

Guided checkpoint

A student says Tr(AB)=5\operatorname{Tr}(AB)=5 for the matrices in the preceding problem. What is the correct value?

Source: Chapter 2, section 2.10, pp. 46-47

Section 05

Determinants and volume to Derive the first principal direction

01

Determinants and volume

A zero eigenvalue collapses at least one direction and makes the determinant zero, which is precisely the singular case.

Guided checkpoint

Judge each statement.

Source: Chapter 2, section 2.11, pp. 47-48

02

Derive the first principal direction

Constrained to unit length, the best one-dimensional reconstruction direction is the leading eigenvector of the data covariance or, equivalently, the leading right singular vector of the centered data matrix.

Guided checkpoint

For centered data and a unit vector ww, which optimization identifies the first principal component direction in the chapter's reconstruction view?

Source: Chapter 2, section 2.12, pp. 48-52

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.