What Is Bayes' Theorem? (Definition)
The theorem was formulated by the Reverend Thomas Bayes (1701–1761) and published posthumously in 1763. The French mathematician Pierre-Simon Laplace independently derived and generalized it around 1812, which is why it occasionally appears as the Bayes-Laplace theorem in older texts. The underlying logic is straightforward: your belief about an event should be updated proportionally to how well the observed evidence fits that event compared to all alternative explanations.
In the language of the formula: P(A) is the prior — what you believed before observing anything. P(B|A) is the likelihood — how probable the evidence is if A is true. P(B) is the marginal probability of the evidence across all possible causes. The result, P(A|B), is the posterior — your updated belief after seeing the evidence. For a complete grounding in the prerequisite concepts, the conditional probability guide on Statistics Fundamentals covers P(A|B) notation in detail.
- Prior probability P(A): Your initial estimate of A before observing any evidence (e.g., disease prevalence in the population)
- Likelihood P(B|A): The probability of observing evidence B, given that A is true (e.g., test sensitivity)
- Marginal probability P(B): The total probability of the evidence under all scenarios — computed via the law of total probability
- Posterior probability P(A|B): The updated probability of A after seeing evidence B — this is the answer you're solving for
- Base rate neglect: The most common error — ignoring P(A) and treating the likelihood as the posterior
The Bayes' Theorem Formula
P(A|B) = posterior probability
P(B|A) = likelihood
P(A) = prior probability
P(B) = marginal probability of evidence
Because P(B) is often not given directly, it is expanded using the law of total probability. For a binary hypothesis (A or not-A, written Ac):
P(Aᶜ) = 1 − P(A)
P(B|Aᶜ) = false positive rate
This expanded form is the one you will use in almost every real problem, because you are usually given sensitivity (P(B|A)) and specificity (1 − P(B|Aᶜ)) rather than P(B) itself. The 4-step method below uses this form throughout.
How to Solve Bayes' Theorem Problems — 4-Step Method
Step 1: Identify P(A) — the prior (base rate). Step 2: Find P(B|A) — the likelihood (how probable is the evidence if A is true?). Step 3: Compute P(B) using the law of total probability. Step 4: Apply the formula P(A|B) = P(B|A) × P(A) / P(B) to get the posterior. Interpret in plain language.
Define the Prior Probability P(A)
The prior is the base rate — how common is the event A in the relevant population before any test or observation? In medical examples this is prevalence. In spam filtering it is the fraction of incoming email that is spam. Getting this right is the most important step; ignoring the prior causes base rate neglect, the most frequent error in probabilistic reasoning.
Find the Likelihood P(B|A) and P(B|Aᶜ)
The likelihood P(B|A) is the probability of observing evidence B given that A is true. In a medical test this is the sensitivity. You also need P(B|Aᶜ) — how probable is the same evidence when A is false (the false-positive rate). In a test context, P(B|Aᶜ) = 1 − specificity.
Calculate P(B) Using the Law of Total Probability
P(B) = P(B|A) × P(A) + P(B|Aᶜ) × P(Aᶜ). This denominator is simply the total probability of seeing the evidence, regardless of whether A is true or not. Once you have P(B), you have everything needed for the final step.
Compute the Posterior P(A|B) and Interpret
Divide the numerator P(B|A) × P(A) by the denominator P(B). The result is the posterior probability — your updated belief in A after seeing evidence B. Always state the conclusion in plain language: "Given a positive test result, the probability the patient actually has the disease is X%."
10 Bayes' Theorem Examples — Fully Solved
Each example below follows the 4-step method. The arithmetic is shown in full. Probability trees accompany the medical and fraud examples where visual layout most aids understanding. All base rates come from published epidemiological or industry sources cited at the end of each example.
Example 1 — Medical Disease Testing (The Classic Example)
Problem: A disease affects 1% of the population. A diagnostic test has 99% sensitivity (true positive rate) and 95% specificity (true negative rate). A randomly selected patient tests positive. What is the probability they actually have the disease?
Prior: P(Disease) = 0.01, so P(No Disease) = 0.99
Likelihoods:
P(Positive | Disease) = 0.99 — sensitivity
P(Positive | No Disease) = 1 − 0.95 = 0.05 — false positive rate
Total probability of a positive test:
P(Positive) = (0.99 × 0.01) + (0.05 × 0.99)
P(Positive) = 0.0099 + 0.0495 = 0.0594
Apply Bayes' theorem:
P(Disease | Positive) = 0.0099 / 0.0594 = 0.1667 ≈ 16.7%
✅ Conclusion: Despite a 99% accurate test, a positive result only means a 16.7% chance of actually having the disease. This counterintuitive result stems from base rate neglect — the disease is rare (1%), so false positives from the 99% of healthy people vastly outnumber true positives from the 1% who are sick.
Probability Tree — Disease Testing Example
Example 2 — Email Spam Filter
Problem: 40% of emails received are spam. The word "FREE" appears in 80% of spam emails and in 10% of legitimate emails. An email contains the word "FREE." What is the probability it is spam?
Prior: P(Spam) = 0.40, P(Legitimate) = 0.60
Likelihoods:
P("FREE" | Spam) = 0.80
P("FREE" | Legitimate) = 0.10
P("FREE"):
= (0.80 × 0.40) + (0.10 × 0.60) = 0.32 + 0.06 = 0.38
Posterior:
P(Spam | "FREE") = (0.80 × 0.40) / 0.38 = 0.32 / 0.38 = 0.842 ≈ 84.2%
✅ Conclusion: An email containing "FREE" has an 84.2% probability of being spam. This is the foundational logic behind the Naive Bayes classifier, which extends this calculation across dozens of words simultaneously. See the full treatment in Bayesian Machine Learning.
Example 3 — Credit Card Fraud Detection
Problem: 0.5% of credit card transactions are fraudulent. A fraud alert system flags 90% of actual fraud cases (sensitivity = 0.90) but also triggers on 2% of legitimate transactions (false positive rate = 0.02). Given that a transaction is flagged, what is the probability it is truly fraudulent?
Prior: P(Fraud) = 0.005, P(Legitimate) = 0.995
Likelihoods:
P(Flag | Fraud) = 0.90
P(Flag | Legitimate) = 0.02
P(Flag):
= (0.90 × 0.005) + (0.02 × 0.995) = 0.0045 + 0.0199 = 0.0244
Posterior:
P(Fraud | Flag) = 0.0045 / 0.0244 = 0.1844 ≈ 18.4%
✅ Conclusion: Even with a 90% sensitive detection system, only 18.4% of flagged transactions are actually fraudulent. This is why banks perform secondary verification rather than automatically blocking every flagged card — the vast majority of alerts would wrongly freeze legitimate accounts.
📊 Original Dataset — Fraud Detection Across 10,000 Transactions
| Actual Status | Transactions | Flagged (Alert) | Not Flagged |
|---|---|---|---|
| Fraudulent (0.5%) | 50 | 45 (true positives) | 5 (missed) |
| Legitimate (99.5%) | 9,950 | 199 (false positives) | 9,751 (true negatives) |
| Total Flagged | 10,000 | 244 | 9,756 |
P(Fraud | Flagged) = 45 / 244 = 18.4% — matches the Bayesian calculation exactly. Original dataset constructed for this guide.
Example 4 — Weather Forecast Accuracy
Problem: It rains 30% of days in a city in autumn. When rain actually occurs, a weather model correctly predicts it 85% of the time. When it does not rain, the model incorrectly predicts rain 20% of the time. The model has predicted rain today. What is the probability it will actually rain?
Prior: P(Rain) = 0.30, P(No Rain) = 0.70
Likelihoods:
P(Predict Rain | Rain) = 0.85
P(Predict Rain | No Rain) = 0.20
P(Predict Rain):
= (0.85 × 0.30) + (0.20 × 0.70) = 0.255 + 0.140 = 0.395
Posterior:
P(Rain | Predicted Rain) = 0.255 / 0.395 = 0.6456 ≈ 64.6%
✅ Conclusion: Despite a reasonably accurate forecast model, "Rain predicted" translates to only a 64.6% probability of actual rain — not 85%. The 20% false positive rate on non-rain days (which make up 70% of days) pulls the posterior down substantially. This is why calibrated probabilistic forecasts are more informative than binary yes/no predictions.
Example 5 — Manufacturing Defect Detection
Problem: A factory produces machine parts with a 2% defect rate. An automated inspection system catches 95% of defective parts (sensitivity = 0.95) and incorrectly rejects 3% of good parts (false positive rate = 0.03). If the system rejects a part, what is the probability the part is genuinely defective?
Prior: P(Defective) = 0.02, P(Good) = 0.98
Likelihoods:
P(Reject | Defective) = 0.95
P(Reject | Good) = 0.03
P(Reject):
= (0.95 × 0.02) + (0.03 × 0.98) = 0.0190 + 0.0294 = 0.0484
Posterior:
P(Defective | Reject) = 0.0190 / 0.0484 = 0.3926 ≈ 39.3%
✅ Conclusion: Only 39.3% of rejected parts are actually defective. This means roughly 6 out of every 10 rejections are false alarms. A plant manager must weigh the cost of discarding good parts against the cost of shipping defective ones — Bayes' theorem quantifies exactly how much of each type of error is occurring.
Example 6 — Workplace Drug Testing
Problem: 5% of employees in a company use a prohibited substance. A urine test has 98% sensitivity and 97% specificity. An employee tests positive. What is the probability they actually use the substance?
Prior: P(User) = 0.05, P(Non-User) = 0.95
Likelihoods:
P(Positive | User) = 0.98
P(Positive | Non-User) = 1 − 0.97 = 0.03
P(Positive):
= (0.98 × 0.05) + (0.03 × 0.95) = 0.049 + 0.0285 = 0.0775
Posterior:
P(User | Positive) = 0.049 / 0.0775 = 0.6323 ≈ 63.2%
✅ Conclusion: There is a 63.2% chance the employee is actually a user — meaning a 36.8% chance the positive test is a false alarm. This illustrates why confirmatory testing (a second, different test) is standard practice before taking disciplinary action. Related concepts are covered in sensitivity and specificity.
Example 7 — University Admissions Prediction
Problem: A university admits 30% of applicants. Among admitted students, 85% had a GPA above 3.5. Among rejected applicants, 40% had a GPA above 3.5. An applicant has a GPA above 3.5. What is the probability they will be admitted?
Prior: P(Admitted) = 0.30, P(Rejected) = 0.70
Likelihoods:
P(GPA > 3.5 | Admitted) = 0.85
P(GPA > 3.5 | Rejected) = 0.40
P(GPA > 3.5):
= (0.85 × 0.30) + (0.40 × 0.70) = 0.255 + 0.280 = 0.535
Posterior:
P(Admitted | GPA > 3.5) = 0.255 / 0.535 = 0.4766 ≈ 47.7%
✅ Conclusion: A high GPA raises the admission probability from 30% (base rate) to 47.7% — a meaningful increase, but far from guaranteed. Because many rejected applicants also have high GPAs, the evidence only partially updates the prior. This is Bayesian reasoning in practice: evidence shifts beliefs without necessarily settling the question.
Example 8 — E-Commerce Purchase Prediction
Problem: 15% of website visitors make a purchase. Among buyers, 70% had viewed the product page at least three times. Among non-buyers, 20% had also viewed the product page three or more times. A visitor has viewed the page three or more times. What is the probability they will purchase?
Prior: P(Buyer) = 0.15, P(Non-Buyer) = 0.85
Likelihoods:
P(3+ views | Buyer) = 0.70
P(3+ views | Non-Buyer) = 0.20
P(3+ views):
= (0.70 × 0.15) + (0.20 × 0.85) = 0.105 + 0.170 = 0.275
Posterior:
P(Buyer | 3+ views) = 0.105 / 0.275 = 0.3818 ≈ 38.2%
✅ Conclusion: Repeated page views raise the purchase probability from 15% to 38.2%. This is the logic behind triggered email campaigns and retargeting ads — Bayesian inference identifies which behavioral signals most strongly update the prior toward a purchase. Connecting this to conditional probability reveals how each additional signal can be chained to further refine the estimate.
Example 9 — Naive Bayes Text Classification
Problem: A news classifier labels articles as "Sports" or "Politics." 40% of training articles are Sports. The word "goal" appears in 60% of Sports articles and 5% of Politics articles. The word "election" appears in 15% of Sports articles and 80% of Politics articles. A new article contains both words. Assuming independence (Naive Bayes assumption), which class is more probable?
Priors: P(Sports) = 0.40, P(Politics) = 0.60
Likelihoods (both words, independence assumed):
P("goal","election" | Sports) = 0.60 × 0.15 = 0.090
P("goal","election" | Politics) = 0.05 × 0.80 = 0.040
Unnormalized scores (numerators only):
Sports: 0.090 × 0.40 = 0.036
Politics: 0.040 × 0.60 = 0.024
Total: 0.060
Posteriors:
P(Sports | words) = 0.036 / 0.060 = 60%
P(Politics | words) = 0.024 / 0.060 = 40%
✅ Conclusion: The classifier assigns this article to Sports with 60% probability. The word "goal" is much more diagnostic than "election" in tipping the balance. This exact calculation, extended over thousands of words, is how email spam filters (Naive Bayes) and many natural language processing classifiers work. For the full ML context see Bayesian Machine Learning.
Example 10 — Bayesian A/B Testing
Problem: A product team tests two landing page designs. Based on historical data, 70% of new design tests improve conversion rates (prior belief). Design B shows a lift in a preliminary sample: given this lift, A/B testing records show that 80% of truly better designs produce a statistically significant lift, while 30% of inferior designs also produce a significant lift by chance. Design B shows a significant lift. What is the updated probability Design B is genuinely better?
Prior: P(Better) = 0.70, P(Not Better) = 0.30
Likelihoods:
P(Significant Lift | Better) = 0.80
P(Significant Lift | Not Better) = 0.30
P(Significant Lift):
= (0.80 × 0.70) + (0.30 × 0.30) = 0.560 + 0.090 = 0.650
Posterior:
P(Better | Significant Lift) = 0.560 / 0.650 = 0.8615 ≈ 86.2%
✅ Conclusion: A statistically significant lift raises the probability of a genuine improvement from 70% (prior) to 86.2%. This is Bayesian A/B testing — rather than a binary "significant / not significant" decision, you get a continuously updated probability. For frequentist alternatives see the Bayesian vs Frequentist comparison.
The BAYES Framework — An Original Decision Guide
The following mnemonic was developed by the Statistics Fundamentals team to make the 4-step process memorable across any application domain. Each letter corresponds to an irreducible part of the calculation.
The BAYES Framework — 5 Steps to Any Bayesian Problem
Begin with the Prior
State P(A) — the base rate before any evidence. Never skip this step; it is the most common source of error.
Assess the Evidence
Identify both likelihoods: P(B|A) and P(B|Aᶜ). Be precise about what "evidence" means in the problem context.
Yield the Denominator
Compute P(B) = P(B|A)·P(A) + P(B|Aᶜ)·P(Aᶜ) using the law of total probability.
Execute the Formula
Divide the numerator [P(B|A) × P(A)] by the denominator P(B) to get the posterior P(A|B).
State the Conclusion
Translate the number back into plain language. Explain whether the evidence substantially moved the prior and what the practical implications are.
Real-World Applications of Bayes' Theorem
Bayes' theorem appears across nearly every discipline where decisions are made under uncertainty. The applications below represent areas where the theorem provides direct, quantifiable guidance rather than a loose analogy.
Clinical Diagnosis
Posterior probabilities guide whether a positive test result warrants treatment, watchful waiting, or confirmatory testing. Used in cancer screening, HIV testing, and genetic risk assessment.
Spam & Content Filtering
Naive Bayes classifiers scan incoming messages for word frequencies and compute the posterior probability of spam. Gmail's initial spam filter was Bayesian at its core.
Fraud Detection
Banks update transaction-fraud probabilities in real time using behavioral priors. Unusual location, amount, or merchant category each shift the posterior via Bayes' theorem.
Machine Learning
Bayesian inference underpins probabilistic classifiers, hyperparameter optimization, and uncertainty quantification in deep learning. Libraries including SciPy and PyMC implement these methods.
Weather Forecasting
Numerical weather models update probability distributions over atmospheric states as new observations arrive — a continuous Bayesian update over time.
Search Engine Ranking
Language models and ranking algorithms use Bayesian priors over user intent to interpret ambiguous queries and select the most probable relevant documents.
Quality Control
Statistical process control uses Bayesian updating to decide when a production line has drifted out of specification, minimizing both false alarms and missed defects.
A/B Testing
Bayesian A/B testing returns a posterior probability that variant B outperforms A, allowing decisions at any sample size without fixed stopping rules.
Comparison Tables
Bayes' Theorem vs Conditional Probability
| Aspect | Conditional Probability | Bayes' Theorem |
|---|---|---|
| What it computes | P(A|B) directly from a joint probability table | P(A|B) from P(B|A), P(A), and P(B) |
| When to use it | When joint probabilities are known | When only the reverse conditional and the prior are known |
| Requires prior | Not explicitly | Yes — P(A) is essential |
| Key formula | P(A|B) = P(A ∩ B) / P(B) | P(A|B) = P(B|A)·P(A) / P(B) |
| Typical context | Frequency tables, survey data | Medical testing, machine learning, forecasting |
Prior vs Posterior Probability
| Term | Notation | Meaning | Example |
|---|---|---|---|
| Prior | P(A) | Probability before observing evidence | 1% disease prevalence |
| Likelihood | P(B|A) | Probability of the evidence given A is true | 99% test sensitivity |
| Marginal | P(B) | Total probability of the evidence | P(positive test) = 0.0594 |
| Posterior | P(A|B) | Updated probability after evidence | 16.7% probability of disease |
Bayesian vs Frequentist Statistics
| Dimension | Bayesian | Frequentist |
|---|---|---|
| Probability definition | Degree of belief, updated with evidence | Long-run frequency of outcomes |
| Uses prior information | Yes — explicitly encodes prior beliefs | No — data alone drives conclusions |
| Output | Posterior probability distribution | Point estimate + p-value + confidence interval |
| Small sample behavior | Regularized by prior; avoids overfitting | Unreliable; wide confidence intervals |
| Key theorem | Bayes' theorem | Central Limit Theorem, hypothesis testing |
Entity and Formula Glossary
| Term | Symbol | Definition |
|---|---|---|
| Bayes' Theorem | — | P(A|B) = P(B|A)·P(A) / P(B). Updates belief about A after observing B. |
| Prior Probability | P(A) | Probability of hypothesis A before observing any evidence. |
| Posterior Probability | P(A|B) | Updated probability of A after evidence B is observed. |
| Likelihood | P(B|A) | Probability of observing evidence B given that A is true. |
| Marginal Probability | P(B) | Total probability of evidence B under all hypotheses. |
| Conditional Probability | P(A|B) | Probability of A given that B is known to have occurred. |
| Joint Probability | P(A ∩ B) | Probability that both A and B occur simultaneously. |
| Probability Tree | — | Branching diagram showing all outcomes and their probabilities at each stage. |
| Bayesian Inference | — | The process of using Bayes' theorem to update probability estimates as new data arrives. |
| Naive Bayes | — | A classifier that extends Bayes' theorem by assuming conditional independence among features. |
| Sensitivity | P(+|Disease) | True positive rate: fraction of actual positives correctly identified by a test. |
| Specificity | P(−|No Disease) | True negative rate: fraction of actual negatives correctly identified. |
| False Positive | P(+|No Disease) | Incorrect positive result when the condition is absent. = 1 − Specificity. |
| False Negative | P(−|Disease) | Incorrect negative result when the condition is present. = 1 − Sensitivity. |
| Base Rate | P(A) | The prevalence or overall frequency of event A in the relevant population. |
| Thomas Bayes | — | English statistician (1701–1761) who first described the theorem, published posthumously in 1763. |
| Pierre-Simon Laplace | — | French mathematician who independently developed and generalized Bayes' theorem around 1812. |
5 Common Misconceptions About Bayes' Theorem
| Misconception | Why It's Wrong | The Correct Interpretation |
|---|---|---|
| "A 99% accurate test means a positive result is 99% reliable." | Ignores the base rate (prior probability) | When the condition is rare, most positives are false alarms — as in Example 1 (16.7% true positive rate) |
| "P(A|B) and P(B|A) are the same thing." | Confusing the two is called the Prosecutor's Fallacy | P(Disease|Positive) ≠ P(Positive|Disease). The formula explicitly shows they are related but distinct. |
| "Bayes' theorem is only for medical testing." | That is just the most famous example | Bayes' theorem applies to any situation with a prior belief updated by new evidence — finance, ML, law, and engineering all use it. |
| "A low prior means the test is useless." | A low prior raises the bar, but does not nullify the update | A very high likelihood ratio can overcome a low prior. Multiple independent tests also multiply the updates. |
| "You need to know exact priors for Bayes' theorem to be useful." | Prior sensitivity analysis handles uncertainty in the prior | Running the calculation with a range of plausible priors shows how robust the posterior is to prior choice. |
Interactive Bayes' Theorem Calculator
Enter your prior, likelihood, and false positive rate below. The calculator shows the full Bayesian update — numerator, denominator, and posterior — so you can trace every step alongside the 10 examples above. For more probability calculations see the Bayes' Theorem Calculator and the broader Probability Calculator.
🔬 Bayes' Theorem Calculator — Step-by-Step
Bayes' Theorem Cheat Sheet
| Formula / Concept | Expression | Notes |
|---|---|---|
| Bayes' theorem | P(A|B) = P(B|A)·P(A) / P(B) | Core formula |
| Expanded denominator | P(B) = P(B|A)·P(A) + P(B|Aᶜ)·P(Aᶜ) | Law of total probability |
| Complement rule | P(Aᶜ) = 1 − P(A) | Always needed in practice |
| Sensitivity | P(+|Disease) | True positive rate |
| Specificity | P(−|No Disease) | True negative rate |
| False positive rate | 1 − Specificity | P(+|No Disease) |
| False negative rate | 1 − Sensitivity | P(−|Disease) |
| Naive Bayes (two features) | P(A|B,C) ∝ P(B|A)·P(C|A)·P(A) | Independence assumed |
| Odds form | Posterior odds = Likelihood ratio × Prior odds | Convenient for updating iteratively |
| Sequential update | Apply Bayes' theorem twice: posterior₁ becomes prior₂ | Each new test refines the estimate |
Frequently Asked Questions
Bayes' theorem is a rule for updating beliefs when new evidence arrives. You start with a prior belief (how likely something is), observe evidence, and use the formula P(A|B) = P(B|A)·P(A)/P(B) to get a new, updated probability called the posterior. The key insight is that the strength of evidence must always be interpreted relative to the starting probability — a strong test still produces many false positives if the underlying event is rare.
The most cited real-life example is medical testing. If a disease affects 1% of people and a test is 99% accurate, a positive result still only means about 16.7% probability of actually having the disease. This counterintuitive result — shown fully in Example 1 above — is why doctors order confirmatory tests rather than treating on a single positive result. Other everyday examples include spam filtering, fraud alerts on credit cards, and weather forecast interpretation.
Conditional probability P(A|B) can be computed directly from a joint probability table: P(A|B) = P(A ∩ B)/P(B). Bayes' theorem is a rearrangement that lets you compute P(A|B) when you know the reverse conditional P(B|A) instead. This is crucial in practice because you typically know the test's performance (how often it detects a condition when it is present) rather than the joint frequency. For the full comparison see the conditional probability guide.
The prior probability P(A) is your initial estimate of how likely event A is, before any test or observation. In medical examples it is disease prevalence. In spam filtering it is the fraction of all incoming email that is spam. The prior is not arbitrary — it should reflect genuine population data or established base rates. Ignoring the prior (treating the likelihood as if it were the posterior) is the most common probabilistic reasoning error, called base rate neglect. See the prior probability guide for a full treatment.
Bayes' theorem is central to machine learning in multiple ways. The Naive Bayes classifier (Example 9 above) computes class posteriors by multiplying word likelihoods with class priors — it classifies text into spam vs legitimate, sentiment vs no sentiment, or topic categories. Bayesian optimization uses a probabilistic surrogate model to efficiently search hyperparameter space. Probabilistic graphical models (Bayesian networks) represent complex joint distributions as products of conditional probabilities. Libraries including SciPy, PyMC, and Stan provide ready-to-use Bayesian inference tools.
The posterior probability P(A|B) is the updated probability of A after observing evidence B. It is the output of Bayes' theorem and represents your revised belief. In the disease testing example, the posterior is the probability the patient has the disease after learning their test is positive — which turned out to be 16.7%, much lower than the 99% test sensitivity might suggest. Read the full posterior probability guide for worked examples with distributions.
Bayes' theorem is not appropriate when: (1) you cannot specify a meaningful prior — choosing a prior arbitrarily can bias results; (2) the events in question are not independent from the way the problem is set up; (3) a frequentist p-value is what a peer reviewer or regulatory body requires. Bayesian methods also tend to be computationally expensive for high-dimensional problems, though Markov chain Monte Carlo methods in PyMC and Stan address this.
Start with a single root node. Draw two branches from it — one for each outcome of the hypothesis (A and Aᶜ) — labelled with P(A) and P(Aᶜ). From each of those nodes, draw two more branches for the evidence occurring or not: P(B|A), P(Bᶜ|A), P(B|Aᶜ), and P(Bᶜ|Aᶜ). Multiply probabilities along each path to get joint probabilities (e.g., P(A ∩ B) = P(A) × P(B|A)). Sum the relevant paths to get P(B), then divide the path you care about by P(B). The visual tool at Probability Tree Diagram draws these automatically.
Sources and References
All formulas and base rates in this guide are cross-referenced against the primary sources below. Worked examples are constructed using these published base rates so the arithmetic can be independently verified.
- Bayes, T. (1763) — "An Essay towards Solving a Problem in the Doctrine of Chances." Philosophical Transactions of the Royal Society, 53, 370–418. Foundation paper.
- Gigerenzen, G. & Hoffrage, U. (1995) — "How to Improve Bayesian Reasoning Without Instruction: Frequency Formats." Psychological Review, 102(4), 684–704. PubMed Central
- OpenStax Introductory Statistics — Ch. 3: Probability Topics. Rice University. openstax.org
- Stanford Encyclopedia of Philosophy — "Bayes' Theorem." plato.stanford.edu
- NIST Engineering Statistics Handbook — Probability and Distributions. itl.nist.gov
- MIT OpenCourseWare — 6.041 Probabilistic Systems Analysis and Applied Probability. ocw.mit.edu
- Khan Academy — Conditional probability and Bayes' theorem. khanacademy.org