Quizmaster

Learning objectives

What you will be able to explain

What information does the Hessian capture?
Compute a gradient and Hessian
What Hessian condition characterizes a twice-differentiable convex function?
Take one Newton step
Compute one momentum update
Match each optimizer to its key idea

Section 01

What information does the Hessian capture? to What Hessian condition characterizes a twice-differentiable convex function?

What information does the Hessian capture?

The Hessian contains second derivatives and describes how curvature changes across directions. This is what makes second-order methods different from plain gradient descent.

Guided checkpoint

Choose the best description.

Compute a gradient and Hessian

The gradient is

(2x + y, x + 4y)

, which becomes

(1, -3)

(1, -1)

. The Hessian is constant:

\begin{bmatrix} 2 & 1 \\ 1 & 4 \end{bmatrix}

Guided checkpoint

Let $f(x, y) = x^2 + xy + 2y^2$ . Compute the gradient at $(1, -1)$ and the distinct Hessian entries.

What Hessian condition characterizes a twice-differentiable convex function?

For twice-differentiable functions, positive semidefinite curvature everywhere is the standard convexity test. This links optimization geometry directly to linear algebra.

Guided checkpoint

Choose the standard criterion.

Section 02

Take one Newton step to Match each optimizer to its key idea

Take one Newton step

Newton's update is

w_{t+1} = w_t - f'(w_t) / f''(w_t)

. Here

f'(w) = 2w - 6

and

f''(w) = 2

, so

w_1 = 0 - (-6)/2 = 3

Guided checkpoint

Use Newton's method on $f(w) = w^2 - 6w$ starting from $w_0 = 0$ . Compute $w_1$ .

Compute one momentum update

The new velocity is

0.9 \cdot 0.2 + 4 = 4.18

. The parameter update is then

1 - 0.1 \cdot 4.18 = 0.582

Guided checkpoint

Use the update $v_{t+1} = \beta v_t + g_t$ and $w_{t+1} = w_t - \eta v_{t+1}$ . Let $v_t = 0.2$ , $g_t = 4$ , $\beta = 0.9$ , $\eta = 0.1$ , and $w_t = 1$ . Compute $v_{t+1}$ and $w_{t+1}$ .

Match each optimizer to its key idea

These methods differ mainly in how they use gradient information and whether they account for noise, momentum, curvature, or parameter-wise scaling.

Guided checkpoint

Match the method to the most characteristic property.

Section 03

Optimization facts: true or false to Approximate a derivative by finite differences

Optimization facts: true or false

Curvature classifies stationary points locally, convexity constrains global structure, and Adam is specifically designed to adapt updates across parameters and time.

Guided checkpoint

Mark each statement as true or false.

What is a saddle point?

At a saddle point, different directions have different curvature behavior. This is one reason high-dimensional optimization landscapes can be tricky even when the gradient is small.

Guided checkpoint

Choose the best description.

Approximate a derivative by finite differences

We compute

f(3.1) = 9.61

and

f(3) = 9

, so the approximation is

(9.61 - 9)/0.1 = 6.1

. The exact derivative would be

2w = 6

w = 3

Guided checkpoint

Use the forward difference approximation $\frac{f(w+h) - f(w)}{h}$ for $f(w) = w^2$ at $w = 3$ with $h = 0.1$ .

Section 04

Why can plain gradient descent zig-zag in narrow valleys?

When one direction is much steeper than another, a fixed learning rate can overreact in one coordinate and underreact in another. That is a hallmark of ill-conditioned optimization problems.

Guided checkpoint

Choose the best explanation.

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course