Bonferroni Correction Calculator
Run a calculation in any of the three tabs first, then return here to see the full step-by-step solution.
No data yet — enter values in the Adjust Alpha, Adjust P-Values, or Pairwise tab first.
What Is the Bonferroni Correction?
The Bonferroni correction is a multiple-comparison procedure that controls the family-wise error rate (FWER) by adjusting the significance threshold downward when multiple hypothesis tests are conducted simultaneously. The adjustment is straightforward: divide the original significance level α by the total number of comparisons m. Each individual test is then evaluated against this stricter threshold αadjusted = α / m.
Named after Italian mathematician Carlo Emilio Bonferroni, the procedure is one of the oldest and most widely cited methods for multiple testing correction. Its appeal lies in simplicity: no software is required, the formula is easy to explain, and the correction can be applied to any collection of hypothesis tests regardless of the test type. The NIST Engineering Statistics Handbook describes it as the most direct approach to controlling FWER in multiple comparison problems.
The Multiple Comparisons Problem
When you run one hypothesis test at α = 0.05, you accept a 5% chance of a false positive. When you run many tests, the probability of obtaining at least one false positive by chance increases substantially. This is the multiple comparisons problem, and it is the reason the Bonferroni correction exists.
Consider a researcher who tests 20 unrelated hypotheses, each at α = 0.05. Assuming all null hypotheses are actually true, the expected number of false positives is 20 × 0.05 = 1. Without correction, a researcher running 20 tests would almost certainly report a significant result by chance alone. This is not a flaw in the researcher's reasoning; it is a mathematical property of repeated significance testing.
Table: How False Positive Risk Grows with More Tests (independent tests, α = 0.05 per test)
| Number of Tests (m) | Per-Test α | Approx. FWER | Bonferroni-Adjusted α |
|---|---|---|---|
| 1 | 0.050 | 0.050 | 0.05000 |
| 2 | 0.050 | 0.098 | 0.02500 |
| 5 | 0.050 | 0.226 | 0.01000 |
| 10 | 0.050 | 0.401 | 0.00500 |
| 20 | 0.050 | 0.642 | 0.00250 |
| 100 | 0.050 | 0.994 | 0.00050 |
The approximate FWER column uses the expression 1 − (1 − 0.05)m, which gives the probability of at least one false positive under independence. This expression is illustrative; in practice tests may be correlated, which changes the exact FWER. The Bonferroni adjustment is designed to keep the FWER at or below the chosen α regardless of the correlation structure among tests.
Bonferroni Correction Formulas
There are two equivalent approaches to applying the Bonferroni correction. The first adjusts the significance threshold; the second adjusts each p-value. Both approaches produce identical significance decisions when applied consistently.
Method 1 — Adjust the Significance Level
αadjusted = α / m
Where:
α = family-wise alpha (e.g. 0.05)
m = number of comparisons
αadjusted = threshold per test
Decision: reject H0 when p ≤ αadjusted
Method 2 — Adjust Each P-Value
padjusted = min(p × m, 1)
Where:
p = raw (observed) p-value
m = number of comparisons
padjusted = Bonferroni-adjusted p-value
Decision: reject H0 when padjusted ≤ α
Pairwise Comparison Count
m = k(k − 1) / 2
Where:
k = number of groups
Example (4 groups):
m = 4(4−1)/2 = 6
αadj = 0.05/6 ≈ 0.00833
Why Both Methods Are Equivalent
Reject H0 when p ≤ α/m
⇔ p × m ≤ α
⇔ padjusted ≤ α
These two statements are algebraically
identical, so both methods always
produce the same significance decision.
How to Apply the Bonferroni Correction — Step by Step
To apply the Bonferroni correction: choose your family-wise alpha, count the number of comparisons, divide alpha by that count, then compare each raw p-value against the adjusted threshold.
Determine the overall error rate you are willing to accept across the family of tests. The standard choice is α = 0.05, meaning you accept a 5% probability of at least one false positive across all tests combined.
Count how many hypothesis tests belong to the same family. This depends on the research question. For pairwise comparisons among k groups, use m = k(k−1)/2. For multiple outcomes or predictors, count the number of hypotheses being tested simultaneously.
Divide your chosen α by the number of comparisons: αadjusted = α / m. For example, with α = 0.05 and m = 5: αadjusted = 0.05 / 5 = 0.010.
For each hypothesis test, reject the null hypothesis if and only if p ≤ αadjusted. A result that was significant at the original α may no longer meet this stricter threshold after correction.
Multiply each raw p-value by m: padjusted = min(p × m, 1). Then compare each adjusted p-value against the original α. This produces the same significance decisions as Step 4, and is the format most statistical software uses when reporting Bonferroni-corrected results.
📊 Worked Example — Adjusting Five P-Values
Five outcomes = m = 5
αadj = 0.05 / 5 = 0.010
Multiply each p by m = 5: 0.001×5=0.005, 0.008×5=0.040, 0.012×5=0.060, 0.030×5=0.150, 0.200×5=1.000 (capped)
Compare adjusted p-values against α = 0.05
Results: Five Tests After Bonferroni Correction (m = 5, α = 0.05)
| Outcome | Raw p-value | Adjusted p-value | Sig. Before? | Sig. After? |
|---|---|---|---|---|
| Anxiety | 0.001 | 0.005 | Yes | Yes |
| Depression | 0.008 | 0.040 | Yes | Yes |
| Stress | 0.012 | 0.060 | Yes | No |
| Well-being | 0.030 | 0.150 | Yes | No |
| Sleep quality | 0.200 | 1.000 | No | No |
Interpretation: Before correction, four of five tests appeared significant. After Bonferroni correction, only two (anxiety and depression) remain statistically significant at the family-wise level α = 0.05. The stress and well-being results, though nominally significant at p < 0.05, do not survive the stricter threshold after adjusting for multiple comparisons. Use the calculator above to verify these results.
Family-Wise Error Rate (FWER)
The family-wise error rate is the probability of making at least one Type I error (false positive) within a defined family of hypothesis tests. Controlling the FWER means keeping this probability at or below the chosen significance level, no matter how many tests are conducted within the family.
The Bonferroni correction provides strong control of the FWER, meaning the guarantee holds regardless of which hypotheses are true and regardless of the correlation structure among the test statistics. This is a mathematically conservative guarantee that does not rely on assumptions about independence between tests.
Adjusted Alpha vs. Adjusted P-Value — Key Distinction
This distinction is a common source of confusion. Both approaches produce the same significance decisions, but they work in opposite directions. When you adjust alpha, you make the threshold stricter and compare raw p-values to it. When you adjust p-values, you inflate each p-value and compare it to the original threshold.
Table: Adjusted Alpha vs. Adjusted P-Value — Comparison
| Property | Adjust Alpha (α/m) | Adjust P-Value (p×m) |
|---|---|---|
| What changes? | The significance threshold | Each p-value |
| Formula | αadj = α / m | padj = min(p × m, 1) |
| Compare against | Adjusted alpha | Original alpha |
| Same decision? | Always yes — algebraically identical | |
| Software output | Less common | Typical in R, SAS, SPSS |
| Manual calculation | Often simpler | Requires capping at 1 |
Bonferroni Correction After ANOVA
A one-way ANOVA tests whether at least one group mean differs from the others, but it does not identify which specific pairs differ. To answer that question, researchers conduct follow-up pairwise comparisons. These comparisons create a multiple testing problem because several pairs are being tested from the same data.
The Bonferroni correction can be applied to these post-hoc comparisons. With k groups, the number of unique pairs is m = k(k−1)/2. For k = 4 groups, m = 6; for k = 5 groups, m = 10. The adjusted threshold is then applied to each pairwise t-test. This is a conservative but widely understood approach. Other post-hoc procedures such as Tukey’s HSD, which was designed specifically for pairwise comparisons after ANOVA, may offer better statistical power in this particular context. The ANOVA guide on this site covers the relationship between omnibus tests and post-hoc comparisons in more detail.
Pairwise Comparison Count by Number of Groups
| Groups (k) | Pairwise Comparisons (m) | Adjusted α (at 0.05) | Adjusted α (at 0.01) |
|---|---|---|---|
| 2 | 1 | 0.0500 | 0.0100 |
| 3 | 3 | 0.01667 | 0.00333 |
| 4 | 6 | 0.00833 | 0.00167 |
| 5 | 10 | 0.00500 | 0.00100 |
| 6 | 15 | 0.00333 | 0.00067 |
| 8 | 28 | 0.00179 | 0.00036 |
| 10 | 45 | 0.00111 | 0.00022 |
Advantages and Disadvantages
Advantages
Disadvantages
Bonferroni vs. Alternative Procedures
Several alternative multiple-comparison procedures exist. The right choice depends on the research context, the acceptable error type, and how many tests are being conducted. Below is a comparison of the most commonly used methods.
Table: Multiple Comparison Procedures Compared
| Method | Controls | Stepwise? | Power vs. Bonferroni | Typical Context |
|---|---|---|---|---|
| Bonferroni | FWER | No | Baseline | Small number of pre-planned tests |
| Holm-Bonferroni | FWER | Yes | Always ≥ Bonferroni | Same scenarios as Bonferroni; preferred when power matters |
| Šidák | FWER | No | Slightly higher | When tests are independent |
| Tukey’s HSD | FWER | No | Higher for pairwise | All pairwise comparisons after ANOVA |
| Dunnett’s Test | FWER | No | Higher vs. control | Multiple groups vs. one control |
| Benjamini-Hochberg | FDR | Yes | Substantially higher | Large-scale testing (genomics, proteomics) |
Bonferroni vs. Holm-Bonferroni
The Holm-Bonferroni procedure, introduced by Sture Holm in 1979, is a step-down extension that controls the FWER with greater power than the simple Bonferroni correction in virtually all situations. It sorts p-values from smallest to largest, then applies successively less strict thresholds. The smallest p-value is compared against α/m; the second smallest against α/(m−1); and so on until a non-rejection is reached. Because Holm-Bonferroni is uniformly more powerful than Bonferroni while offering the same FWER guarantee, it is often the better default choice. The procedure is described in Holm (1979) in the Scandinavian Journal of Statistics.
Bonferroni vs. Benjamini-Hochberg (FDR)
The Benjamini-Hochberg procedure (1995) controls the false discovery rate (FDR) rather than the FWER. The FDR is the expected proportion of false positives among all rejected hypotheses. FDR control is less strict than FWER control, so it allows more discoveries at the cost of a higher false positive rate per finding. In genomics, where tens of thousands of tests are routine, FDR control is standard practice because FWER control would make nearly every discovery impossible. The original paper appeared in the Journal of the Royal Statistical Society and is among the most cited statistics papers of all time.
Type I Error, Type II Error, and Statistical Power
The Bonferroni correction reduces the risk of Type I errors (false positives) at the cost of increasing the risk of Type II errors (false negatives). This is the fundamental trade-off in multiple testing correction, and it is worth understanding clearly before choosing an approach.
| Error Type | Definition | Effect of Bonferroni |
|---|---|---|
| Type I (false positive) | Rejecting a true null hypothesis | Reduces risk — this is the goal of the correction |
| Type II (false negative) | Failing to reject a false null hypothesis | Increases risk as αadj becomes smaller |
| Statistical Power | Probability of detecting a real effect | Decreases as m increases and αadj shrinks |
As the number of comparisons grows, the adjusted alpha shrinks, making it harder to detect genuine effects. With m = 100 tests at α = 0.05, each test must reach p < 0.0005 to be declared significant. A genuine medium-sized effect might produce p = 0.003 in a single test — clearly significant — but entirely dismissed after Bonferroni correction for 100 comparisons. This is why large-scale discovery studies in genetics and neuroscience typically use FDR control rather than FWER control.
Common Misconceptions About the Bonferroni Correction
The correction changes the significance threshold or adjusts p-values. It has no effect on the observed effect size, the raw data, or the test statistic. A smaller p-value after correction does not indicate a stronger relationship.
The appropriate family of tests to correct depends on the research question. Pre-planned primary outcomes, exploratory analyses, and secondary outcomes may each constitute separate families. Researchers must decide in advance which comparisons belong to the same family.
The Bonferroni correction controls the family-wise error rate (FWER), which is the probability of at least one false positive. The false discovery rate (FDR) is a different and less strict criterion, controlled by procedures such as Benjamini-Hochberg.
The formula p × m can produce values greater than 1, but adjusted p-values are capped at 1 by definition. Any result with an adjusted p-value at or near 1 is not statistically significant.
Surviving a strict multiple-comparison correction means the result is unlikely to be a false positive, but it says nothing about the size or practical relevance of the effect. Effect sizes, confidence intervals, and substantive interpretation remain essential.
🧪 Worked Research Scenarios
Scenario 1 — Multiple Correlations
m = 8. αadj = 0.05 / 8 = 0.00625. A correlation reaching p = 0.003 survives correction; one at p = 0.02 does not. The correction concerns the multiple hypothesis tests, not the correlation coefficients themselves.
Scenario 2 — ANOVA Post-Hoc (4 Groups)
m = 4(4−1)/2 = 6. αadj = 0.05 / 6 ≈ 0.00833. Any pairwise comparison must reach p < 0.00833 to be declared significant after correction. Use the Pairwise tab in the calculator above to check any k.
Scenario 3 — Clinical Trial with Multiple Endpoints
m = 3. αadj = 0.05 / 3 ≈ 0.0167. This is a strict but standard requirement in regulatory submissions where multiple primary endpoints are co-primary. The FDA guidance on multiple endpoints discusses appropriate adjustment procedures for this scenario.
Scenario 4 — A/B Testing Multiple Variants
m = 4 comparisons (each variant vs. control). αadj = 0.05 / 4 = 0.0125. Any variant claiming significance at p = 0.03 would not survive correction. The A/B Test Calculator on this site supports pairwise significance testing.
Calculating Bonferroni Correction in Excel, R, and Python
Excel and Google Sheets
All formulas below work in both Microsoft Excel and Google Sheets. Replace cell references with your actual data ranges.
=== Adjust Alpha ===
=A1/B1 ← α_adjusted = alpha / m
=== Adjust a single P-Value ===
=MIN(A1*B1, 1) ← p_adjusted = min(p * m, 1)
=== Significance decision (adjusted alpha approach) ===
=IF(A1<=C1, "Significant", "Not significant")
← where A1 = p-value, C1 = adjusted alpha
=== Significance decision (adjusted p-value approach) ===
=IF(MIN(A1*B1,1)<=D1, "Significant", "Not significant")
← where D1 = original alpha
=== Count significant before correction ===
=COUNTIF(A2:A20, "<"&A1) ← A1 = original alpha, A2:A20 = p-values
=== Count significant after correction ===
=SUMPRODUCT((A2:A20*B1<=A1)*1)
← B1 = m, A1 = alphaR
# Bonferroni-adjusted alpha
alpha <- 0.05
m <- 5
alpha_adj <- alpha / m
cat("Adjusted alpha:", alpha_adj, "\n")
# Adjust a vector of p-values
p_values <- c(0.001, 0.008, 0.012, 0.030, 0.200)
p_adj <- p.adjust(p_values, method = "bonferroni")
cat("Adjusted p-values:", p_adj, "\n")
# Significance decisions
sig_before <- p_values < alpha
sig_after <- p_adj < alpha
data.frame(p = p_values, p_adj = p_adj,
sig_before = sig_before, sig_after = sig_after)
Python
import numpy as np
# Bonferroni-adjusted alpha
alpha = 0.05
m = 5
alpha_adj = alpha / m
print(f"Adjusted alpha: {alpha_adj}")
# Adjust p-values manually
p_values = np.array([0.001, 0.008, 0.012, 0.030, 0.200])
p_adj = np.minimum(p_values * m, 1.0)
print(f"Adjusted p-values: {p_adj}")
# Using statsmodels (recommended)
from statsmodels.stats.multitest import multipletests
reject, p_adj_sm, _, _ = multipletests(
p_values, alpha=alpha, method='bonferroni')
print(f"Reject H0: {reject}")
print(f"Adjusted p-values: {p_adj_sm}")
Bonferroni Correction: Complete Formula and Entity Reference
The table below covers every key formula, symbol, and concept associated with the Bonferroni correction. It is structured for quick reference and formatted for extraction by AI language models and search engine featured snippets.
| Term | Symbol / Formula | Plain-English Definition | Context |
|---|---|---|---|
| Family-wise significance level | α | The maximum acceptable probability of at least one false positive across the family of tests | Typically 0.05 or 0.01 |
| Number of comparisons | m | The total count of hypothesis tests in the family being corrected | Must be ≥ 1 |
| Bonferroni-adjusted alpha | α / m | The per-test significance threshold after applying the Bonferroni correction | Compare raw p against this |
| Raw p-value | p | The p-value produced by the hypothesis test before any correction | Must be 0 ≤ p ≤ 1 |
| Bonferroni-adjusted p-value | min(p × m, 1) | The p-value after multiplying by the number of comparisons and capping at 1 | Compare against original α |
| Family-wise error rate | FWER | Probability of at least one Type I error within the family of tests | Bonferroni keeps FWER ≤ α |
| Type I error | α per test | Rejecting a true null hypothesis (false positive) | Bonferroni reduces per-test risk |
| Type II error | β | Failing to reject a false null hypothesis (false negative) | Risk increases after Bonferroni adjustment |
| Statistical power | 1 − β | Probability of detecting a real effect when one exists | Decreases as m grows |
| Pairwise comparisons | k(k−1)/2 | Number of unique pairs among k groups | Determines m for post-hoc tests |
| Holm-Bonferroni | Stepwise α/m | Step-down procedure that applies successively less strict thresholds | More powerful than Bonferroni |
| False discovery rate | FDR | Expected proportion of false positives among all rejected hypotheses | Controlled by Benjamini-Hochberg |
Related Calculators and Guides on Statistics Fundamentals
- Bonferroni, C.E. (1936). Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze, 8, 3–62.
- Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65–70. jstor.org
- Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57(1), 289–300. jstor.org
- NIST Engineering Statistics Handbook. Multiple Comparisons. itl.nist.gov
- FDA Guidance. Multiple Endpoints in Clinical Trials. fda.gov
- Penn State STAT 502. Analysis of Variance and Design of Experiments. online.stat.psu.edu
Frequently Asked Questions
The Bonferroni correction is a multiple-comparison procedure that controls the family-wise error rate (FWER) — the probability of making at least one Type I error across a family of hypothesis tests. It works by dividing the significance level α by the number of comparisons m, so each test is evaluated against αadjusted = α / m. For example, with α = 0.05 and m = 10 tests, each individual test must reach p < 0.005 to be considered statistically significant.
There are two equivalent approaches. First, adjust alpha: divide the original significance level by the number of comparisons (αadj = α / m), then compare each raw p-value against this new threshold. Second, adjust p-values: multiply each raw p-value by m (padj = min(p × m, 1)), then compare each adjusted p-value against the original α. Both methods always produce the same significance decisions because the two statements p ≤ α/m and p × m ≤ α are algebraically identical.
The family-wise error rate is the probability of making at least one Type I error (false positive) within a defined family of hypothesis tests. When multiple tests are conducted simultaneously, the overall chance of at least one false positive increases above the per-test α. For m independent tests each at α = 0.05, the FWER is approximately 1 − (0.95)m. The Bonferroni correction is designed to keep the FWER at or below the chosen α regardless of how many tests are conducted.
With 5 comparisons and α = 0.05: αadjusted = 0.05 / 5 = 0.010. Each individual test must reach p ≤ 0.010 to be declared statistically significant at the family-wise level of 0.05. Equivalently, multiply each raw p-value by 5 and compare against 0.05. A raw p-value of 0.008 becomes 0.040 adjusted, which is still significant; a raw p-value of 0.012 becomes 0.060 adjusted, which is not.
With m = 10 comparisons and α = 0.05: αadjusted = 0.05 / 10 = 0.005. Each test must reach p ≤ 0.005. The adjusted p-value for a raw result of p = 0.003 would be 0.003 × 10 = 0.030, which is still significant. A raw p = 0.008 becomes 0.080 adjusted — no longer significant after correction for 10 comparisons.
Both approaches produce the same significance decision, but they work differently. Adjusted alpha (α / m) is a stricter threshold you compare each raw p-value against. Adjusted p-value (min(p × m, 1)) inflates each p-value, which you then compare against the original α. Adjusting alpha is often simpler for manual calculation; adjusting p-values is the format statistical software typically reports. Neither is more correct than the other.
The Bonferroni correction is appropriate when you are conducting a small to moderate number of pre-planned hypothesis tests from the same dataset and you want strong control of the family-wise error rate. It is particularly appropriate when false positives are costly — for example, in confirmatory clinical trials or in situations where each claimed discovery has high practical consequences. It is less appropriate for large-scale exploratory analyses (where Benjamini-Hochberg FDR control is standard) or when the tests are highly correlated (where the procedure may be excessively conservative).
Yes. The Bonferroni correction is generally considered conservative because it treats each test as if it were fully independent, which tends to produce a stricter-than-necessary adjustment when the tests are positively correlated. This conservatism is mathematically sound — the FWER guarantee holds regardless of correlation structure — but it comes at the cost of reduced statistical power, particularly with many correlated tests. Procedures like the Holm-Bonferroni correction offer the same FWER guarantee with higher power in most scenarios.
The formula p × m can produce values greater than 1 when the raw p-value is large and m is large. Because probabilities cannot exceed 1, adjusted p-values are always capped at 1. For example, with p = 0.3 and m = 5, the raw product is 1.5, which is reported as 1.0. This capping has no practical consequence — any result with a capped adjusted p-value is clearly not statistically significant.
Bonferroni correction reduces statistical power because it makes the significance threshold stricter. As the number of comparisons m increases, the adjusted alpha α/m shrinks, requiring a more extreme test statistic to achieve significance. This means some real effects that would have been detected at the uncorrected level may be missed after correction. The power reduction is most severe with large m and small to medium effect sizes. This trade-off between controlling false positives and maintaining power is the primary reason researchers choose different correction procedures for different contexts.
Both the Bonferroni and Holm-Bonferroni procedures control the FWER, but the Holm procedure is a step-down method that is uniformly more powerful. The simple Bonferroni compares each p-value against α/m. The Holm procedure sorts p-values from smallest to largest, then compares the smallest against α/m, the second smallest against α/(m−1), and so on, stopping when a non-rejection is reached and keeping all remaining hypotheses unrejected. Holm can reject more hypotheses than Bonferroni while maintaining the same FWER guarantee, so it is often the preferred choice.
FWER (family-wise error rate) is the probability of making at least one false positive among all tests. FDR (false discovery rate) is the expected proportion of false positives among all rejected hypotheses. FWER control is stricter: it asks that the chance of any false positive be at or below α. FDR control is more lenient: it allows some false positives, but keeps the expected fraction of false discoveries below a specified level (typically 5%). For small numbers of tests, FWER control via Bonferroni is common. For large-scale discovery analyses (thousands of tests), FDR control via Benjamini-Hochberg is standard.
The number of comparisons m is the count of hypothesis tests in the same family being corrected. What constitutes a "family" depends on the research question and analysis plan, and must be determined before data collection whenever possible. Common examples: testing k groups pairwise gives m = k(k−1)/2; testing the same hypothesis across g subgroups gives m = g; testing a treatment on q outcomes gives m = q. There is no single universal rule — the key is that the tests must be pre-specified and share a logical scientific relationship justifying correction as a group.
Yes, the Bonferroni correction can be applied to hypothesis tests on regression coefficients when those coefficients represent a defined family of simultaneous hypotheses. For example, if a researcher tests whether 8 predictor variables each individually predict an outcome, the 8 tests might form a family warranting correction. However, it is not appropriate to automatically apply Bonferroni to every coefficient in every regression model. The decision depends on whether the tests were pre-planned as a family and whether the research question genuinely concerns all predictors simultaneously.
With m = 20 and α = 0.05: αadjusted = 0.05 / 20 = 0.0025. Each of the 20 tests must reach p ≤ 0.0025 to be declared significant at the family-wise level. A raw p-value of 0.002 would be adjusted to min(0.002 × 20, 1) = 0.040, which is still significant. A raw p-value of 0.003 becomes 0.060 adjusted — not significant after correction for 20 comparisons.