QUIZ
Tree-based models
9 questions on Tree-based models, from 4 lessons. Pick an answer in your head, then open it.
On the roadmap for: ML Engineering, Data Science.
Question 1 of 9
How does a classification tree choose a split?
- A. It splits on the features in the order they appear in the data
- B. It picks the feature with the most distinct values, since that splits the most
- C. It picks the split that most reduces impurity, such as Gini or entropy
- D. It chooses a random feature and threshold at each node
View answer
✓ C. It picks the split that most reduces impurity, such as Gini or entropy
Each split greedily makes the child nodes as pure as possible. Regression trees reduce variance instead.
Lesson: Decision Trees
Question 2 of 9
Why does a very deep decision tree often do badly on new data?
- A. It uses too few features at each split to generalise
- B. It averages too many leaves together and blurs the data
- C. It memorises noise in the training set (high variance)
- D. It can't fit the training data well enough to be useful
View answer
✓ C. It memorises noise in the training set (high variance)
With enough depth every training point gets its own leaf. Limit depth or leaf size, or prune.
Lesson: Decision Trees
Question 3 of 9
What makes the trees in a random forest different from each other?
- A. Each tree sees a different time period of the data
- B. Each tree is trained with a different loss function
- C. Each uses a bootstrap sample and random feature subsets at splits
- D. Each tree has a different maximum depth, but all of them see the same data
View answer
✓ C. Each uses a bootstrap sample and random feature subsets at splits
Both sources of randomness decorrelate the trees, so averaging them cuts variance.
Lesson: Random Forest
Question 4 of 9
What is a random forest's out-of-bag error?
- A. The error on the rows each tree didn't see in its bootstrap sample
- B. The error on a separate held-out test set that none of the trees were trained on
- C. The error of the single worst tree in the forest
- D. The error on the training rows each tree was fitted to
View answer
✓ A. The error on the rows each tree didn't see in its bootstrap sample
About a third of the rows are left out of each bootstrap, so they act as a free validation set.
Lesson: Random Forest
Question 5 of 9
In gradient boosting, what does each new tree learn?
- A. The average prediction of all the trees built so far
- B. The original labels, with each new tree voting equally
- C. A random subset of the labels, drawn the same way that bagging draws its rows
- D. The negative gradient of the loss: the residuals, for squared error
View answer
✓ D. The negative gradient of the loss: the residuals, for squared error
Each tree corrects the ensemble's current mistakes, and a learning rate scales its contribution.
Lesson: Gradient Boosting
Question 6 of 9
What happens if you keep adding trees to gradient boosting with no early stopping?
- A. It can't overfit, because each tree in it is kept small
- B. It turns into a random forest once there are enough trees
- C. It can overfit; a random forest with more trees mostly just stabilises
- D. It always underfits once there are more than a few hundred trees in the ensemble
View answer
✓ C. It can overfit; a random forest with more trees mostly just stabilises
Boosting keeps fitting the residuals, eventually including noise. Use a small learning rate plus early stopping.
Lesson: Gradient Boosting
Question 7 of 9
Which statement about bagging and boosting is correct?
- A. Bagging mainly reduces bias; boosting mainly reduces variance
- B. Bagging mainly reduces variance; boosting mainly reduces bias
- C. Both mainly reduce variance
- D. Both mainly reduce bias
View answer
✓ B. Bagging mainly reduces variance; boosting mainly reduces bias
Bagging averages independent strong learners; boosting adds weak learners in sequence to fix errors.
Question 8 of 9
When does combining models help the most?
- A. When the models are accurate and their errors aren't correlated
- B. When you train many copies of the same model on exactly the same data
- C. When the base models are weaker than random guessing
- D. When the models make the same mistakes on the same cases
View answer
✓ A. When the models are accurate and their errors aren't correlated
Averaging cancels independent errors. Models that fail on the same cases add nothing.
Question 9 of 9
In stacking, what should the meta-model be trained on?
- A. The raw features, without the base models' predictions
- B. The base models' predictions on their own training rows
- C. The base models' predictions on the test set
- D. Out-of-fold predictions from the base models
View answer
✓ D. Out-of-fold predictions from the base models
Predictions on data a base model was trained on are over-optimistic; out-of-fold ones avoid that leakage.