QUIZ
ML core concepts
10 questions on ML core concepts, from 7 lessons. Pick an answer in your head, then open it.
On the roadmap for: AI Engineering, ML Engineering, Data Science.
Question 1 of 10
Your model has high error on both training and validation data. What is the most likely issue?
- A. Data leakage between training and validation
- B. High variance: the model is overfitting the training data
- C. Too much training data for the model's size
- D. High bias: the model is too simple to fit the data
View answer
✓ D. High bias: the model is too simple to fit the data
If it can't even fit the training data, add capacity or better features, not more data.
Lesson: The Bias–Variance Trade-off
Question 2 of 10
Training error is low but validation error is much higher. Which fix fits?
- A. Add regularisation or more training data
- B. Train for more epochs until validation improves
- C. Remove the validation set and train on everything
- D. Make the model bigger so it can fit better
View answer
✓ A. Add regularisation or more training data
A big gap between train and validation is variance. Regularisation, more data or a simpler model reduce it.
Lesson: The Bias–Variance Trade-off
Question 3 of 10
Which of these causes data leakage?
- A. Keeping a separate held-out test set untouched until the end
- B. Splitting into folds first, then fitting the scaler inside each one
- C. Shuffling the training rows before each epoch so batches differ every time
- D. Fitting the scaler or picking features on all data before cross-validation
View answer
✓ D. Fitting the scaler or picking features on all data before cross-validation
Preprocessing must be fitted inside each training fold; otherwise information from the validation fold leaks in.
Lesson: Cross-Validation & Data Leakage
Question 4 of 10
How should you cross-validate a model that forecasts next month's sales?
- A. Leave-one-out, so the model trains on almost all of the data
- B. Validate on a random 20% of months chosen from anywhere in the whole history
- C. Forward-chaining: train on the past, validate on the period after it
- D. Random k-fold with shuffling, so every month is used for training
View answer
✓ C. Forward-chaining: train on the past, validate on the period after it
Shuffling lets the model see the future. Time-ordered splits match how it will be used.
Lesson: Cross-Validation & Data Leakage
Question 5 of 10
Only 2% of transactions are fraud. Which change needs no retraining and directly trades precision for recall?
- A. Moving the decision threshold
- B. SMOTE
- C. Collecting more fraud examples
- D. Class weights
View answer
✓ A. Moving the decision threshold
Lowering the threshold flags more cases (higher recall, lower precision) using the same model's scores.
Lesson: Class Imbalance
Question 6 of 10
Why does random search often beat grid search for the same number of training runs?
- A. Grid search can't run trials in parallel
- B. Random search doesn't need a validation set
- C. It covers every combination, so it can never miss the best one on the grid
- D. Few hyperparameters usually matter, and it tries more values of each
View answer
✓ D. Few hyperparameters usually matter, and it tries more values of each
A 5×5 grid tries only 5 values of the important parameter; 25 random trials try 25.
Lesson: Hyperparameter Tuning
Question 7 of 10
On a learning curve, training and validation error have converged at a high value. Will more data help?
- A. Yes: more data always reduces validation error in the end
- B. Yes, as long as you at least double the amount of training data you have
- C. Only if you also train for many more epochs on the new data
- D. Probably not: it's underfitting and needs more capacity or features
View answer
✓ D. Probably not: it's underfitting and needs more capacity or features
Converged curves at high error mean bias. More data helps when there is still a gap between the curves.
Question 8 of 10
What do SHAP values give you for one prediction?
- A. The model's overall accuracy on similar customers
- B. How much the training loss fell when each feature was first added to the model
- C. Each feature's contribution, summing to this prediction minus the average
- D. Which features the model ignored for this prediction
View answer
✓ C. Each feature's contribution, summing to this prediction minus the average
SHAP comes from Shapley values: contributions that sum exactly to prediction minus the base value.
Lesson: Explaining Models: SHAP, Permutation Importance and PDPs
Question 9 of 10
Two features are strongly correlated. What can happen to their permutation importance?
- A. Both look very important, because each carries the same signal as the other
- B. Their importance becomes negative, because they cancel out
- C. Nothing; permutation importance handles correlation correctly
- D. Both can look unimportant, since the model leans on the other one
View answer
✓ D. Both can look unimportant, since the model leans on the other one
The model still has the information through the partner feature, so each one's importance is understated.
Lesson: Explaining Models: SHAP, Permutation Importance and PDPs
Question 10 of 10
A logistic model outputs log-odds of 0 for a customer. What probability is that?
- A. 0.5
- B. 1
- C. It depends on the features
- D. 0
View answer
✓ A. 0.5
Probability = 1 / (1 + e⁻⁰) = 0.5. Log-odds of 0 means even odds.
Lesson: Classical ML Practice