AIML Rounds

DATA SCIENCE · ROUND 1 OF 3

Statistics and probability

The interviewer checks that you reason clearly about uncertainty, tests and evidence, and can explain them to non-experts.

8 questions · suggested time 40 min
  1. Question 1 of 8

    A fraud flag is 99% accurate, and 0.5% of transactions are fraud. A transaction is flagged. How likely is it to be fraud?

    Answer out loud first, then open the model answer.

    View model answer

    I'd use Bayes' rule, and counts make it easy. Out of 100,000 transactions, 500 are fraud and the flag catches about 495 of them. Of the 99,500 genuine ones, 1% are wrongly flagged, about 995. So about 495 out of 1,490 flags are fraud, roughly 33%.

    The point is the base rate: when the event is rare, even an accurate test produces mostly false alarms. In practice I'd tune the threshold and maybe add a second check before acting.

    What the interviewer listens for

    • Uses Bayes / natural frequencies
    • Gets roughly one in three
    • Explains the base-rate effect
    • Practical next step

    Lessons: Probability Basics & Bayes' Rule, Bayes' Rule Practice

  2. Question 2 of 8

    Explain a p-value to a product manager, and say what it doesn't mean.

    Answer out loud first, then open the model answer.

    View model answer

    If the change truly did nothing, the p-value is how often we'd see a difference at least as big as ours just by chance. A small p-value means our result would be surprising if there were no real effect.

    It isn't the probability the change works, and it says nothing about how big or valuable the effect is. A tiny, useless effect can be significant with enough users, so I always report the effect size with a confidence interval.

    What the interviewer listens for

    • Correct definition under the null
    • Not P(hypothesis is true)
    • Not effect size or importance
    • Pairs it with a confidence interval

    Lessons: Hypothesis Tests, p-values & A/B Testing

  3. Question 3 of 8

    What does a 95% confidence interval mean, and when would you use the bootstrap to get one?

    Answer out loud first, then open the model answer.

    View model answer

    If we repeated the study many times and built an interval the same way each time, about 95% of those intervals would contain the true value. It describes the method, not a 95% chance for this one interval.

    I'd use the bootstrap when there's no simple formula, for a median, a ratio or a skewed metric like revenue: resample the data with replacement many times, recompute the statistic, and take the middle 95% of the results.

    What the interviewer listens for

    • Repeated-sampling interpretation
    • Avoids the 'probability for this interval' trap
    • Bootstrap = resample with replacement
    • When: medians, ratios, skewed metrics

    Lessons: Confidence Intervals & the Bootstrap

  4. Question 4 of 8

    Explain the central limit theorem and why it matters in practice.

    Answer out loud first, then open the model answer.

    View model answer

    The average of many independent samples is approximately normally distributed, whatever the shape of the original data, as long as its variance is finite, and its spread shrinks like σ/√n.

    That's why we can build confidence intervals and run z- or t-tests on averages such as conversion rates, even when individual values are skewed. With heavy tails, like revenue, convergence is slow, so we need more data or robust methods.

    What the interviewer listens for

    • Sample mean → approximately normal
    • Spread shrinks with √n
    • Justifies tests and intervals on means
    • Caveat: heavy tails converge slowly

    Lessons: Law of Large Numbers and Central Limit Theorem

  5. Question 5 of 8

    When would you use a t-test, a chi-squared test, or a non-parametric test?

    Answer out loud first, then open the model answer.

    View model answer

    A t-test compares means of a numeric metric between two groups, a paired t-test when the same units are measured twice. A chi-squared test compares counts across categories, such as conversion yes/no by variant. ANOVA compares means across more than two groups.

    When assumptions fail, for example tiny samples with heavy skew, I'd use a non-parametric test such as Mann-Whitney, or a permutation test that shuffles labels to build the null distribution.

    What the interviewer listens for

    • t-test for means; paired when repeated
    • Chi-squared for categorical counts
    • ANOVA for many groups
    • Non-parametric / permutation when assumptions fail

    Lessons: T-Tests and Z-Tests, Chi-Squared Tests and ANOVA, Non-Parametric and Permutation Tests

  6. Question 6 of 8

    Users who use feature X retain better. Can we say X causes retention?

    Answer out loud first, then open the model answer.

    View model answer

    Not from that alone. Engaged users are more likely both to find X and to stay, so engagement is a confounder. The correlation could even reverse inside groups, as in Simpson's paradox.

    The cleanest answer is an experiment. If we can't run one, I'd use methods such as difference-in-differences around a rollout, matching on prior engagement, or an instrumental variable, and state the assumptions each one needs.

    What the interviewer listens for

    • Names confounding (self-selection)
    • Mentions Simpson's paradox or reverse causation
    • Experiment first
    • Observational methods with stated assumptions

    Lessons: Correlation vs Causation, Causal Inference Without an Experiment

  7. Question 7 of 8

    How many test samples do you need to measure a model's accuracy to within ±2 percentage points?

    Answer out loud first, then open the model answer.

    View model answer

    The standard error of an accuracy p from n samples is √(p(1−p)/n). For a 95% interval of ±2 points, 1.96 × √(p(1−p)/n) ≤ 0.02. In the worst case, p = 0.5, that gives n ≈ 0.25 × (1.96 / 0.02)² ≈ 2,400.

    If accuracy is near 90%, p(1−p) is smaller and about 900 samples suffice. Halving the margin needs four times the samples.

    What the interviewer listens for

    • Uses the standard error of a proportion
    • Worst case p = 0.5 → about 2,400
    • Adjusts for the expected accuracy
    • Halving the margin needs 4× the data

    Lessons: Concentration Inequalities, Confidence Intervals & the Bootstrap

  8. Question 8 of 8

    Bayesian or frequentist: what's the difference, and when would you choose each?

    Answer out loud first, then open the model answer.

    View model answer

    Frequentist methods treat the true value as fixed and describe how procedures behave over repeated samples: p-values, confidence intervals. Bayesian methods treat unknowns as uncertain, combine a prior with the data, and give a posterior, so you can say 'there's an 85% chance B is better'.

    I'd use Bayesian when I have useful prior knowledge, small data, or stakeholders who want direct probability statements; frequentist when I need standard, well-understood error control, as in most A/B testing platforms.

    What the interviewer listens for

    • Fixed parameter vs probability distribution over it
    • Prior + data → posterior
    • Direct probability statements
    • Sensible criteria for choosing

    Lessons: Bayesian vs Frequentist, Bayesian Inference and Conjugate Priors