AIML Rounds

QUIZ

Tree-based models

9 questions on Tree-based models, from 4 lessons. Pick an answer in your head, then open it.

On the roadmap for: ML Engineering, Data Science.

  1. Question 1 of 9

    How does a classification tree choose a split?

    1. A. It splits on the features in the order they appear in the data
    2. B. It picks the feature with the most distinct values, since that splits the most
    3. C. It picks the split that most reduces impurity, such as Gini or entropy
    4. D. It chooses a random feature and threshold at each node
    View answer

    ✓ C. It picks the split that most reduces impurity, such as Gini or entropy

    Each split greedily makes the child nodes as pure as possible. Regression trees reduce variance instead.

    Lesson: Decision Trees

  2. Question 2 of 9

    Why does a very deep decision tree often do badly on new data?

    1. A. It uses too few features at each split to generalise
    2. B. It averages too many leaves together and blurs the data
    3. C. It memorises noise in the training set (high variance)
    4. D. It can't fit the training data well enough to be useful
    View answer

    ✓ C. It memorises noise in the training set (high variance)

    With enough depth every training point gets its own leaf. Limit depth or leaf size, or prune.

    Lesson: Decision Trees

  3. Question 3 of 9

    What makes the trees in a random forest different from each other?

    1. A. Each tree sees a different time period of the data
    2. B. Each tree is trained with a different loss function
    3. C. Each uses a bootstrap sample and random feature subsets at splits
    4. D. Each tree has a different maximum depth, but all of them see the same data
    View answer

    ✓ C. Each uses a bootstrap sample and random feature subsets at splits

    Both sources of randomness decorrelate the trees, so averaging them cuts variance.

    Lesson: Random Forest

  4. Question 4 of 9

    What is a random forest's out-of-bag error?

    1. A. The error on the rows each tree didn't see in its bootstrap sample
    2. B. The error on a separate held-out test set that none of the trees were trained on
    3. C. The error of the single worst tree in the forest
    4. D. The error on the training rows each tree was fitted to
    View answer

    ✓ A. The error on the rows each tree didn't see in its bootstrap sample

    About a third of the rows are left out of each bootstrap, so they act as a free validation set.

    Lesson: Random Forest

  5. Question 5 of 9

    In gradient boosting, what does each new tree learn?

    1. A. The average prediction of all the trees built so far
    2. B. The original labels, with each new tree voting equally
    3. C. A random subset of the labels, drawn the same way that bagging draws its rows
    4. D. The negative gradient of the loss: the residuals, for squared error
    View answer

    ✓ D. The negative gradient of the loss: the residuals, for squared error

    Each tree corrects the ensemble's current mistakes, and a learning rate scales its contribution.

    Lesson: Gradient Boosting

  6. Question 6 of 9

    What happens if you keep adding trees to gradient boosting with no early stopping?

    1. A. It can't overfit, because each tree in it is kept small
    2. B. It turns into a random forest once there are enough trees
    3. C. It can overfit; a random forest with more trees mostly just stabilises
    4. D. It always underfits once there are more than a few hundred trees in the ensemble
    View answer

    ✓ C. It can overfit; a random forest with more trees mostly just stabilises

    Boosting keeps fitting the residuals, eventually including noise. Use a small learning rate plus early stopping.

    Lesson: Gradient Boosting

  7. Question 7 of 9

    Which statement about bagging and boosting is correct?

    1. A. Bagging mainly reduces bias; boosting mainly reduces variance
    2. B. Bagging mainly reduces variance; boosting mainly reduces bias
    3. C. Both mainly reduce variance
    4. D. Both mainly reduce bias
    View answer

    ✓ B. Bagging mainly reduces variance; boosting mainly reduces bias

    Bagging averages independent strong learners; boosting adds weak learners in sequence to fix errors.

    Lesson: Ensembles: Voting, Stacking and Bagging vs Boosting

  8. Question 8 of 9

    When does combining models help the most?

    1. A. When the models are accurate and their errors aren't correlated
    2. B. When you train many copies of the same model on exactly the same data
    3. C. When the base models are weaker than random guessing
    4. D. When the models make the same mistakes on the same cases
    View answer

    ✓ A. When the models are accurate and their errors aren't correlated

    Averaging cancels independent errors. Models that fail on the same cases add nothing.

    Lesson: Ensembles: Voting, Stacking and Bagging vs Boosting

  9. Question 9 of 9

    In stacking, what should the meta-model be trained on?

    1. A. The raw features, without the base models' predictions
    2. B. The base models' predictions on their own training rows
    3. C. The base models' predictions on the test set
    4. D. Out-of-fold predictions from the base models
    View answer

    ✓ D. Out-of-fold predictions from the base models

    Predictions on data a base model was trained on are over-optimistic; out-of-fold ones avoid that leakage.

    Lesson: Ensembles: Voting, Stacking and Bagging vs Boosting