Quizmaster

Learning objectives

What you will be able to explain

Why is the chain rule central to backpropagation?
Run a forward pass through a tiny ReLU network
What is the derivative of the sigmoid function?
Compute a softmax probability
Compute a cross-entropy loss value
Differentiate a one-neuron squared-loss model

Section 01

Why is the chain rule central to backpropagation? to What is the derivative of the sigmoid function?

Why is the chain rule central to backpropagation?

A neural network is a composition of functions. Backpropagation repeatedly applies the chain rule to move gradients from the loss back through each intermediate computation.

Guided checkpoint

Choose the best explanation.

Run a forward pass through a tiny ReLU network

The hidden preactivations are

z_1 = 1

and

z_2 = -1 + 4 - 1 = 2

. ReLU leaves both unchanged, and the output becomes

2 \cdot 1 - 2 + 1 = 1

Guided checkpoint

Let the input be $(x_1, x_2) = (1, 2)$ . Hidden unit 1 has $z_1 = 1 \cdot x_1 + 0 \cdot x_2 + 0$ , hidden unit 2 has $z_2 = -1 \cdot x_1 + 2 \cdot x_2 - 1$ , and both activations are ReLU. The output is $\hat{y} = 2a_1 - a_2 + 1$ . Compute $z_1$ , $a_1$ , $z_2$ , $a_2$ , and $\hat{y}$ .

What is the derivative of the sigmoid function?

The sigmoid derivative can be written in terms of its own output. This compact form is useful in backpropagation derivations.

Guided checkpoint

Choose the correct formula for $\sigma'(z)$ .

Section 02

Compute a softmax probability to Differentiate a one-neuron squared-loss model

Compute a softmax probability

The probability is

\frac{e^2}{e^2 + e^0} = \frac{e^2}{e^2 + 1} \approx 0.8808

. Softmax converts logits into a normalized probability distribution.

Guided checkpoint

For logits $(2, 0)$ , compute the softmax probability of the first class.

Compute a cross-entropy loss value

The cross-entropy loss for the correct class is the negative log of its predicted probability. Here

-\log(0.8) \approx 0.22314

Guided checkpoint

For a one-hot target on the correct class with predicted probability $0.8$ , compute the cross-entropy loss $-\log(0.8)$ using natural logarithms.

Differentiate a one-neuron squared-loss model

The prediction is

2

. The output error is

(\hat{y} - y) = -3

, so the gradients are

-3 \cdot x = -6

for the weight and

-3

for the bias.

Guided checkpoint

Consider a neuron $\hat{y} = wx + b$ with input $x = 2$ , weight $w = 1$ , bias $b = 0$ , target $y = 5$ , and loss $L = \frac{1}{2}(\hat{y} - y)^2$ . Compute $\hat{y}$ , $\partial L / \partial w$ , and $\partial L / \partial b$ .

Section 03

In what order does backpropagation compute gradients? to Match each neural-network quantity to its role

In what order does backpropagation compute gradients?

Backprop starts at the loss because that is where the objective is defined. Gradients are then propagated backward to earlier computations using local derivatives.

Guided checkpoint

Choose the best description.

Backpropagation facts: true or false

These are standard structural facts about neural-network differentiation. Biases are not multiplied by the input, and backprop generalizes to arbitrarily deep differentiable computation graphs.

Guided checkpoint

Mark each statement as true or false.

Match each neural-network quantity to its role

Keeping these roles straight is essential when doing forward-pass bookkeeping and backward-pass differentiation by hand.

Guided checkpoint

Match the term to the best description.

Section 04

Why do sigmoid or tanh networks sometimes suffer from vanishing gradients? to Take one gradient step on a single neuron

Why do sigmoid or tanh networks sometimes suffer from vanishing gradients?

When sigmoid or tanh activations saturate, their local derivatives approach zero. Multiplying many small derivatives across layers can shrink gradients dramatically.

Guided checkpoint

Choose the best explanation.

Take one gradient step on a single neuron

Gradient descent subtracts the gradient:

w_{new} = 1 - 0.1(-6) = 1.6

and

b_{new} = 0 - 0.1(-3) = 0.3

Guided checkpoint

Using the gradients from the earlier neuron example, update $w = 1$ and $b = 0$ with learning rate $0.1$ . The gradients are $\partial L / \partial w = -6$ and $\partial L / \partial b = -3$ .

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course