Quizmaster

Learning objectives

What you will be able to explain

Regression formulation
Design-matrix discipline
Normal-equation estimator
Fit a line exactly
Least squares as MLE
Regularized regression

Section 01

Regression formulation to Normal-equation estimator

Regression formulation

Linear regression is linear in parameters, with features collected in the design matrix.

Guided checkpoint

Match object in $y=Xtheta+\epsilon$ .

Source: Sec. 9.1, pp. 291-292

Design-matrix discipline

Nonlinear basis functions remain linear regression when parameters enter linearly.

Guided checkpoint

Judge each statement.

Source: Sec. 9.1, pp. 291-292

Normal-equation estimator

Stationarity of squared error yields

X^TXtheta=X^Ty

Guided checkpoint

For full-column-rank $X$ , OLS gives:

Source: Sec. 9.2, pp. 292-303

Section 02

Fit a line exactly to Regularized regression

Fit a line exactly

The first point fixes

a=1

and the second gives

5=1+2b

Guided checkpoint

Fit $y=a+bx$ through points $(0,1)$ and $(2,5)$ .

Source: Sec. 9.2, pp. 292-303

Least squares as MLE

The Gaussian negative log-likelihood is quadratic in residuals.

Guided checkpoint

Assume i.i.d. Gaussian observation noise.

Source: Sec. 9.2, pp. 292-303

Regularized regression

L_2

shrinkage stabilizes inversion; exact sparsity is associated more with

L_1

Guided checkpoint

For ridge regression.

Source: Sec. 9.2, pp. 292-303

Section 03

Bayesian linear regression to Maximum likelihood as projection

Bayesian linear regression

Gaussian conjugacy yields analytic posterior and predictive distributions.

Guided checkpoint

Match quantity.

Source: Sec. 9.3, pp. 303-313

Posterior predictive uncertainty

Bayesian prediction integrates parameter uncertainty rather than only plugging in a point.

Guided checkpoint

Judge each claim.

Source: Sec. 9.3, pp. 303-313

Maximum likelihood as projection

The residual is orthogonal to every design-matrix column.

Guided checkpoint

OLS fitted values are best described as:

Source: Sec. 9.4, pp. 313-315

Section 04

Regression geometry to Exercise-style residual check

Regression geometry

Projection need not interpolate when

y

lies outside the column space.

Guided checkpoint

At an OLS solution.

Source: Sec. 9.4, pp. 313-315

Further-reading bridge

Basis expansion and Bayesian regression lead naturally to kernel methods.

Guided checkpoint

Which extension handles nonlinear function spaces through kernels or priors?

Source: Sec. 9.5, p. 315

Exercise-style residual check

Estimate is mean 3; residuals are -1 and 1, with squared norm 2.

Guided checkpoint

For $X=[1,1]^T$ , $y=[2,4]^T$ , fit the intercept-only OLS model. What is the residual squared norm?

Source: Chapter 9 synthesis, pp. 291-315

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course