← Mathematics for Machine Learning

Learn / Mathematics for Machine Learning

07 - Continuous Optimization

Advanced mastery of Chapter 7, covering every named section and exercise-style synthesis across book pages 225-248.

Learning path

0%

0 of 4 sections marked complete · about 42 minutes

Learning objectives

What you will be able to explain

  • Gradient-descent update
  • Gradient-descent behavior
  • One gradient step
  • Constrained optimization
  • A constrained optimum
  • Convex sets and functions

Section 01

Gradient-descent update to One gradient step

01

Gradient-descent update

The negative gradient is steepest local descent in Euclidean geometry.

Guided checkpoint

Which update minimizes a differentiable objective locally?

Source: Sec. 7.1, pp. 227-233

02

Gradient-descent behavior

Curvature and scaling govern stable step sizes.

Guided checkpoint

Judge each claim.

Source: Sec. 7.1, pp. 227-233

03

One gradient step

Gradient is 2(x−3)=−62(x-3)=-6; update gives .6.6.

Guided checkpoint

For f(x)=(x−3)2f(x)=(x-3)^2, start x=0x=0 with η=.1\eta=.1. Compute the next iterate.

Source: Sec. 7.1, pp. 227-233

Section 02

Constrained optimization to Convex sets and functions

01

Constrained optimization

Lagrange multipliers convert equality-constrained stationarity into a coupled system.

Guided checkpoint

Match object.

Source: Sec. 7.2, pp. 233-236

02

A constrained optimum

Stationarity makes x=yx=y; feasibility then gives one each.

Guided checkpoint

Minimize x2+y2x^2+y^2 subject to x+y=2x+y=2. What is the optimizer?

Source: Sec. 7.2, pp. 233-236

03

Convex sets and functions

Convexity turns local information into global guarantees.

Guided checkpoint

Judge each statement.

Source: Sec. 7.3, pp. 236-246

Section 03

Convexity criteria to Convex quadratic

01

Convexity criteria

Equivalent tests depend on differentiability assumptions.

Guided checkpoint

Match criterion.

Source: Sec. 7.3, pp. 236-246

02

Optimality discipline

Optimality combines primal feasibility, dual structure, and stationarity.

Guided checkpoint

For constrained convex problems.

Source: Secs. 7.2-7.3, pp. 233-246

03

Convex quadratic

The Hessian is AA.

Guided checkpoint

When is f(x)=1/2xTAx+bTxf(x)=1/2 x^TAx+b^Tx convex for symmetric AA?

Source: Sec. 7.3, pp. 236-246

Section 04

Exact quadratic minimizer to Exercise-style Hessian test

01

Exact quadratic minimizer

Set derivative 2x−4=02x-4=0; positive curvature confirms the minimum.

Guided checkpoint

Minimize f(x)=x2−4x+7f(x)=x^2-4x+7.

Source: Sec. 7.3, pp. 236-246

02

Further-reading bridge

The chapter is an entry point to convex analysis and numerical optimization.

Guided checkpoint

Which field studies algorithms and duality for convex problems in depth?

Source: Sec. 7.4, p. 246

03

Exercise-style Hessian test

Hessian is [[2,4],[4,10]][[2,4],[4,10]] with determinant 20−16=420-16=4, and positive leading minor.

Guided checkpoint

For f(x,y)=x2+4xy+5y2f(x,y)=x^2+4xy+5y^2, what is the determinant of its Hessian?

Source: Chapter 7 exercises, pp. 247-248

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course