QUIZ
Sequence models
3 questions on Sequence models, from 3 lessons. Pick an answer in your head, then open it.
On the roadmap for: ML Engineering.
Question 1 of 3
Why can't a plain RNN remember a word from 50 steps ago?
- A. It reads every sentence backwards by default
- B. Gradients shrink as they pass back through each step
- C. Its vocabulary is too small to cover long, varied passages of text
- D. It has no memory of earlier steps at all
View answer
✓ B. Gradients shrink as they pass back through each step
Backprop through time repeatedly multiplies by the same weights, so signals fade or explode.
Lesson: RNNs and Backprop Through Time
Question 2 of 3
How does an LSTM keep information for hundreds of steps?
- A. Gates add to a cell state, so memory fades less
- B. It uses attention over all the earlier steps
- C. It never forgets anything it has ever seen
- D. It has many more parameters than a plain RNN
View answer
✓ A. Gates add to a cell state, so memory fades less
The forget, input and output gates decide what to keep, add and expose; GRUs simplify this.
Lesson: LSTM and GRU
Question 3 of 3
What problem did attention (Bahdanau) fix in seq2seq translation?
- A. Slow tokenisation of long input sentences
- B. A single fixed vector for the whole source sentence
- C. Sentences that were too short to translate well
- D. Translating between too many languages at once
View answer
✓ B. A single fixed vector for the whole source sentence
Long sentences lost detail through the bottleneck; attention removed it, and later led to the Transformer.
Lesson: Seq2seq and Bahdanau Attention