Quizmaster

Learning objectives

What you will be able to explain

Why unfold a recurrent graph?
RNN input-output mappings
Compute a scalar recurrence
RNN likelihood and BPTT
When bidirectionality is appropriate
Encoder-decoder components

Section 01

Why unfold a recurrent graph? to Compute a scalar recurrence

Why unfold a recurrent graph?

Unfolding converts a cyclic specification into a finite DAG for a particular sequence, making forward computation and back-propagation through time explicit.

Guided checkpoint

What does unfolding through time reveal?

Source: Chapter 10, section 10.1, pp. 375-378

RNN input-output mappings

Recurrent architectures support several alignments between observed inputs and desired outputs.

Guided checkpoint

Match each task to the natural sequence mapping.

Source: Chapter 10, section 10.2, pp. 378-394

Compute a scalar recurrence

h_1=2

h_2=1

, and

h_3=0.5(1)+1=1.5

Guided checkpoint

Let $h_t=0.5h_{t-1}+x_t$ , $h_0=0$ , and inputs $(x_1,x_2,x_3)=(2,0,1)$ . Compute $h_3$ .

Source: Chapter 10, section 10.2, pp. 378-394

Section 02

RNN likelihood and BPTT to Encoder-decoder components

RNN likelihood and BPTT

Shared transition parameters allow length generalization, while the unfolded graph provides the ordinary reverse-mode gradient calculation.

Guided checkpoint

Judge each statement.

Source: Chapter 10, section 10.2, pp. 378-394

When bidirectionality is appropriate

Bidirectional models combine past and future context, so they suit complete observed sequences but violate causal online constraints.

Guided checkpoint

Which task most naturally permits a bidirectional RNN?

Source: Chapter 10, section 10.3, pp. 394-396

Encoder-decoder components

The fixed context vector in early sequence-to-sequence systems can become an information bottleneck for long inputs.

Guided checkpoint

Match each component to its role.

Source: Chapter 10, section 10.4, pp. 396-398

Section 03

Ways to make an RNN deep to Long-term gradient magnitude

Ways to make an RNN deep

Depth can appear across time, across stacked recurrent layers, or inside the per-step transition and output mappings.

Guided checkpoint

Evaluate each claim.

Source: Chapter 10, section 10.5, pp. 398-400

Sequence recurrence versus structural recursion

Recursive networks generalize recurrent composition from sequences to hierarchical structures such as parse trees.

Guided checkpoint

What distinguishes a recursive neural network?

Source: Chapter 10, section 10.6, pp. 400-401

Long-term gradient magnitude

The product is

0.8^{10}=0.1073741824

, illustrating exponential decay when repeated Jacobian singular values are below one.

Guided checkpoint

A scalar recurrent Jacobian equals 0.8 at each of 10 steps. Ignoring other paths, what multiplicative factor connects a gradient across all 10 steps?

Source: Chapter 10, section 10.7, pp. 401-404

Section 04

Vanishing and exploding gradients to LSTM gate logic

Vanishing and exploding gradients

Gradient dynamics and forward-state dynamics are related but not identical. Clipping addresses explosion, while gates and better paths address persistent credit flow.

Guided checkpoint

Mark each statement as true or false.

Source: Chapter 10, sections 10.7 and 10.11, pp. 401-404 and 413-416

Strategies for multiple time scales

These methods create slow state components or easier credit paths. Learned gates generalize a fixed leak rate.

Guided checkpoint

Match each mechanism to its approach.

Source: Chapter 10, sections 10.8-10.10, pp. 404-413

LSTM gate logic

The nearly additive cell-state path lets an LSTM learn when to preserve, erase, write, and reveal information.

Guided checkpoint

Match each LSTM component to its primary role.

Source: Chapter 10, section 10.10, pp. 408-413

Section 05

GRU versus LSTM to Explicit memory architectures

GRU versus LSTM

Both are gated recurrent architectures designed to learn time scales, but they package state and gating differently.

Guided checkpoint

Which statement is most accurate?

Source: Chapter 10, section 10.10, pp. 408-413

Optimization techniques for long dependencies

Long-term credit assignment benefits from controlling explosion while preserving useful signal propagation; zeroing recurrence would destroy memory.

Guided checkpoint

Select all techniques discussed as relevant.

Source: Chapter 10, section 10.11, pp. 413-416

Explicit memory architectures

External memory creates paths for storing and retrieving information without compressing the entire past into one recurrent vector.

Guided checkpoint

Match each element to its role.

Source: Chapter 10, section 10.12, pp. 416-420

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course