DATA SCIENCE · ROUND 3 OF 3
ML for data science
The interviewer checks you can choose, evaluate and explain models for business questions.
Question 1 of 8
We want to predict which customers will churn next month. Walk me through your approach.
Answer out loud first, then open the model answer.
View model answer
I'd define churn precisely and fix a prediction date. Features may only use data from before it: usage trends, support tickets, billing. Labels come from the following month. I'd split by time, training on earlier months and validating on later ones.
I'd start with a simple baseline such as logistic regression, then try gradient boosting. I'd evaluate with PR-AUC and precision at the number of customers the retention team can contact, and check calibration if probabilities drive spending. Finally, I'd explain the drivers with SHAP and test retention offers in an experiment.
What the interviewer listens for
- Precise label and prediction date
- Point-in-time features; time-based split
- Baseline then stronger model
- Metric tied to the team's capacity; experiment on actions
Lessons: Feature Engineering, Cross-Validation & Data Leakage, Classification Metrics
Question 2 of 8
How would you explain to a stakeholder why the model flagged a particular customer?
Answer out loud first, then open the model answer.
View model answer
I'd use SHAP values for that prediction: each feature's contribution, adding up to the difference between this customer's score and the average. Then I'd translate it into plain words: 'mostly because usage dropped 60% and they raised two complaints'.
For the model overall I'd use permutation importance or partial dependence plots, and warn that correlated features can share or hide importance, and that explanations describe the model, not causes.
What the interviewer listens for
- SHAP for one prediction
- Plain-language translation
- Global: permutation importance, PDPs
- Caveats: correlation, not causation
Lessons: Explaining Models: SHAP, Permutation Importance and PDPs
Question 3 of 8
How would you segment customers when you have no labels?
Answer out loud first, then open the model answer.
View model answer
I'd pick features that matter for the business question, such as spend, frequency, recency and categories, scale them, and maybe reduce dimensions with PCA. Then I'd try k-means for round, similar-sized groups, or a Gaussian mixture or DBSCAN if clusters overlap or have odd shapes.
I'd choose the number of clusters using silhouette scores and whether the segments are distinct and actionable, then profile each segment so the business can name and use it.
What the interviewer listens for
- Business-driven features; scaling
- Method choice and its assumptions
- Choosing k with metrics + judgement
- Profiling and actionability
Lessons: k-Means Clustering, Gaussian Mixture Models, Principal Component Analysis
Question 4 of 8
Your regression model's RMSE got much worse but MAE barely changed. What happened, and which metric should you report?
Answer out loud first, then open the model answer.
View model answer
RMSE squares errors, so a few very large misses dominate it, while MAE weighs all errors equally. The new model probably makes a handful of big errors, perhaps on outliers or a new segment.
Which to report depends on the cost: if big misses are especially costly, as in inventory, RMSE matters; if every unit of error costs the same, MAE. I'd look at the large errors directly to see what caused them.
What the interviewer listens for
- RMSE sensitive to large errors
- Diagnosis: a few big misses
- Metric choice follows business cost
- Inspect the large errors
Lessons: Regression Metrics
Question 5 of 8
Forecast next month's daily sales. What do you do, and how do you know your model is any good?
Answer out loud first, then open the model answer.
View model answer
I'd start with baselines: last week's value for the same weekday (seasonal naive) and a moving average. Then I'd try exponential smoothing or ARIMA, or gradient boosting with lag, calendar and holiday features.
Evaluation must respect time: rolling-origin backtests where each fold trains on the past and forecasts the next period. The model only earns its place if it beats the seasonal naive baseline on those backtests.
What the interviewer listens for
- Seasonal naive baseline
- Classical and ML options
- Rolling, time-ordered backtests
- Must beat the baseline
Lessons: Time-Series Forecasting Basics, Stationarity, ARIMA and Exponential Smoothing
Question 6 of 8
We have no examples of fraud at all. How could you still find suspicious transactions?
Answer out loud first, then open the model answer.
View model answer
I'd use anomaly detection: model what normal looks like and flag what doesn't fit. Options include isolation forests, which isolate unusual points in few random splits, density-based methods, or reconstruction error from an autoencoder.
Anomalies aren't automatically fraud, so I'd send the top-scored cases to analysts, collect their verdicts as labels, and move to a supervised model once I have enough.
What the interviewer listens for
- Anomaly detection framing
- Concrete methods (isolation forest etc.)
- Anomaly ≠ fraud; human review
- Bootstrap labels → supervised model
Lessons: Anomaly Detection
Question 7 of 8
A coupon test shows no overall effect. Is there anything more you can do?
Answer out loud first, then open the model answer.
View model answer
An average of zero can hide groups that respond positively and others that respond negatively or would have bought anyway. Uplift modelling estimates the effect of the coupon for each customer from the experiment data, so we can target the persuadable ones.
Because this is exploring after the fact, I'd validate the targeting rule in a new experiment before rolling it out, and watch for cannibalisation of full-price sales.
What the interviewer listens for
- Average hides heterogeneous effects
- Uplift / CATE modelling
- Target persuadables, not likely buyers
- Validate in a new test
Question 8 of 8
Your model's predicted probabilities feed a pricing decision. How do you check you can trust them?
Answer out loud first, then open the model answer.
View model answer
Good ranking isn't enough; the probabilities must be calibrated, meaning events predicted at 30% happen about 30% of the time. I'd check with a reliability plot and the Brier score or expected calibration error on held-out data.
If they're off, I'd recalibrate with Platt scaling or isotonic regression on a separate set, and recheck calibration over time, since drift can break it.
What the interviewer listens for
- Calibration vs ranking
- Reliability plot, Brier, ECE
- Platt or isotonic recalibration
- Monitor over time
Lessons: Probability Metrics