Probability Basics & Bayes' Rule
A 99% accurate fraud flag fires. Is it really fraud?
The situation
Your fraud alarm catches 90% of fraud and wrongly flags 5% of legitimate orders, and 1% of orders are fraud. A teammate says a flagged order is 90% likely to be fraud. The interviewer asks:
Why can an accurate test for a rare event still produce mostly false positives?
Running example: the food-delivery app's fraud alarm. 1% of orders are fraudulent. The alarm catches 90% of fraud (sensitivity) and wrongly flags 5% of legitimate orders (false-positive rate). An order just triggered the alarm. How likely is it to be fraud? (Practise more in Bayes' Rule Practice.)
Reasoning backwards from evidence
We usually know how evidence behaves given the truth (the alarm's accuracy was measured on known fraud). We need the reverse: the truth given the evidence. Getting this wrong is common enough to have a name, the base-rate fallacy, and it's behind many over-confident ML decisions: "the model is 95% accurate, so a positive is 95% likely to be right" is false for rare events.
The idea
Think of a guard at a packed stadium looking for a few pickpockets. Even a good guard stops plenty of innocent people, simply because there are so many more of them. Bayes' rule is how you count both groups before you trust the flag.
Play
Try to break it. Raise sensitivity from 90% to 99%. Then put it back and cut false alarms from 5% to 1%.
Natural frequencies make Bayes obvious
Each square is one order (expected counts). Red: fraud; teal: legit; orange outline: flagged. Raise the fraud rate to 10% or cut false alarms to 1% and watch the posterior jump.
Two levers move the answer: the base rate and the false-alarm rate. With a 10% fraud rate the same alarm is right 66.7% of the time; with a 1% fraud rate but a 1% false-alarm rate, 47.6%. Improving sensitivity from 90% to 99% barely helps (16.7%). For rare events, specificity matters more than sensitivity for the precision of alerts. my take
Now say why: why does cutting false alarms help so much more than catching more fraud?
Maths
In one breath
A probability is a number between 0 and 1 for how likely an event is. Conditional probability P(A | B) restricts attention to the cases where B happened. Independence means knowing B tells you nothing about A. Bayes' rule turns P(B | A), which is often easy to measure ("how often does a fraudulent order trigger the alarm?"), into P(A | B), which is what you want ("this order triggered the alarm: is it fraud?"). The key ingredient people forget is the base rate, the prior P(A): when the event is rare, even an accurate test produces mostly false alarms. The easiest way to get Bayes right under pressure is to imagine 10,000 cases and count.What you'll be able to do after this: compute conditional probabilities, apply Bayes' rule by counting, explain why base rates dominate for rare events, and connect it to precision, Naive Bayes and Bayesian updating.Joint, conditional, independent
The law of total probability
Sum over every way B can happen. Here P(flag) = 0.009 + 0.0495 = 0.0585: the alarm fires on 5.85% of orders, almost all of them legitimate.
Posterior ∝ likelihood × prior
0.9 × 0.01 / 0.0585 = 0.154. In odds form: posterior odds = prior odds × likelihood ratio = (1/99) × (0.9/0.05) = 18/99 ≈ 0.18, i.e. 15.4%.
In practice
Bayes in ML
- Precision is a posterior: precision = P(actually positive | predicted positive), which collapses for rare classes exactly as above (Classification Metrics, Class Imbalance).
- Naive Bayes classifiers apply Bayes' rule with independent features (Naive Bayes).
- Prior shift: a model trained at one base rate gives wrong probabilities at another; Bayes' rule corrects them (the undersampling correction in Class Imbalance).
- Bayesian inference: priors over parameters, updated by data (MLE & MAP).
The answer that sounds right
"The alarm is 90% accurate, so a flagged order is 90% likely to be fraud"
That confuses P(flag | fraud) = 0.9 with P(fraud | flag). Count 10,000 orders: 100 are fraud and the alarm catches 90 of them. 9,900 are legitimate and 5% of them, 495, are flagged too. Of the 585 flagged orders, only 90 are fraud: 15.4%. The rare base rate means false alarms outnumber real catches more than 5 to 1. Kahneman & Tversky's taxi-cab experiments showed people routinely ignore base rates this way.
Say it
Why can an accurate test for a rare event still produce mostly false positives?
Say your answer out loud
Aim for 30 seconds. It feels different from reading it, and that's the point.
Tick what you covered
The interviewer pushes further
Follow-up 1 of 3
Two tests in a row are positive. What now?
Answer it out loud, then check.
If the tests are conditionally independent given the truth, use the first posterior as the new prior: 0.154 → 0.154 × 0.9 / (0.154 × 0.9 + 0.846 × 0.05) ≈ 0.77.
Follow-up 2 of 3
Is independence the same as mutually exclusive?
Answer it out loud, then check.
No. Mutually exclusive events (can't both happen) are strongly dependent: if one happens, the other's probability drops to 0.
Follow-up 3 of 3
How does this relate to precision and recall?
Answer it out loud, then check.
Recall is P(flag | positive) (sensitivity); precision is P(positive | flag), the Bayes posterior, which depends on prevalence.
The 30-second answer
Bayes' rule says the posterior equals likelihood times prior divided by the evidence: P(A | B) = P(B | A) P(A) / P(B), with P(B) from the law of total probability. The classic trap is ignoring the prior: for a rare event, even an accurate test produces mostly false positives because the many negatives each have a small chance of being flagged. I solve these by imagining a population, say 10,000, and counting. In ML this is why precision collapses on imbalanced data, why Naive Bayes works, and how we correct probabilities when base rates change.
Go further
What not to say
- "P(A | B) = P(B | A)." That's the confusion of the inverse.
- "A 95%-accurate test means a positive is 95% reliable." Depends on the base rate.
Key terms and sources
Events, joint and conditional probability, independence, the law of total probability, Bayes' rule, base rates, and where Bayes shows up in ML · built from Blitzstein & Hwang, Introduction to Probability (ch. 1–2), Deisenroth et al., MML (ch. 6) and Kahneman & Tversky's base-rate studies · checked 28 Oct 2026
Key terms
| Term | Meaning |
|---|---|
| Conditional probability | P(A | B) = P(A ∩ B)/P(B). |
| Independence | P(A ∩ B) = P(A)P(B). |
| Prior / likelihood / posterior | Belief before evidence / P(evidence | hypothesis) / belief after. |
| Base-rate fallacy | Ignoring the prior when judging P(hypothesis | evidence). |
| Likelihood ratio | P(B | A)/P(B | ¬A): how much the evidence shifts the odds. |
Sources
- J. Blitzstein & J. Hwang, Introduction to Probability, 2nd ed., CRC 2019, ch. 1–2, projects.iq.harvard.edu/stat110.
- M. P. Deisenroth, A. A. Faisal & C. S. Ong, Mathematics for Machine Learning, CUP 2020, ch. 6.
- D. Kahneman & A. Tversky, "On the psychology of prediction", Psychological Review 80, 1973 (base-rate neglect; background).