AIML Rounds

QUIZ

Sequence models

3 questions on Sequence models, from 3 lessons. Pick an answer in your head, then open it.

On the roadmap for: ML Engineering.

  1. Question 1 of 3

    Why can't a plain RNN remember a word from 50 steps ago?

    1. A. It reads every sentence backwards by default
    2. B. Gradients shrink as they pass back through each step
    3. C. Its vocabulary is too small to cover long, varied passages of text
    4. D. It has no memory of earlier steps at all
    View answer

    ✓ B. Gradients shrink as they pass back through each step

    Backprop through time repeatedly multiplies by the same weights, so signals fade or explode.

    Lesson: RNNs and Backprop Through Time

  2. Question 2 of 3

    How does an LSTM keep information for hundreds of steps?

    1. A. Gates add to a cell state, so memory fades less
    2. B. It uses attention over all the earlier steps
    3. C. It never forgets anything it has ever seen
    4. D. It has many more parameters than a plain RNN
    View answer

    ✓ A. Gates add to a cell state, so memory fades less

    The forget, input and output gates decide what to keep, add and expose; GRUs simplify this.

    Lesson: LSTM and GRU

  3. Question 3 of 3

    What problem did attention (Bahdanau) fix in seq2seq translation?

    1. A. Slow tokenisation of long input sentences
    2. B. A single fixed vector for the whole source sentence
    3. C. Sentences that were too short to translate well
    4. D. Translating between too many languages at once
    View answer

    ✓ B. A single fixed vector for the whole source sentence

    Long sentences lost detail through the bottleneck; attention removed it, and later led to the Transformer.

    Lesson: Seq2seq and Bahdanau Attention