Quick Comparison: T-Test vs ANOVA at a Glance
A t-test compares the means of exactly two groups using the t-statistic. ANOVA (Analysis of Variance) compares the means of three or more groups using the F-statistic. Both assume normality and homogeneous variance. The key rule: two groups → t-test; three or more groups → ANOVA.
| Factor | T-Test | ANOVA (One-Way) |
|---|---|---|
| Number of groups | Exactly 2 (or 1 vs. population) | 3 or more groups |
| Test statistic | t-statistic | F-statistic |
| Distribution used | t-distribution | F-distribution |
| Degrees of freedom | n₁ + n₂ − 2 (independent) | (k−1) and (N−k) |
| Null hypothesis | μ₁ = μ₂ | μ₁ = μ₂ = … = μₖ |
| Post-hoc tests needed? | No | Yes (if significant) |
| One-tailed test possible? | Yes | No |
| Effect size measure | Cohen's d | Eta squared (η²) or ω² |
| Developed by | Gosset, 1908 | Fisher, 1920s |
What Is a T-Test?
The t-test was published by William Sealy Gosset in 1908 under the pen name "Student" while working at the Guinness Brewery in Dublin. He needed a method that could detect real differences in small samples, where the normal distribution approximation was unreliable. The result was the Student's t-distribution, a bell curve with heavier tails that accounts for the extra uncertainty from estimating the population standard deviation.
There are three main variants. Each addresses a different research design:
One-Sample T-Test
Tests whether a single sample mean differs from a known or hypothesized population value. For example: does the average weight of bags filled by a machine equal the stated 500g? The t-distribution table shows the critical value for the chosen α and degrees of freedom (df = n − 1). Read the full one-sample t-test guide for detailed worked problems.
x̄ = sample mean
μ₀ = hypothesized population mean
s = sample standard deviation
n = sample size
df = n − 1
Independent Samples T-Test (Two-Sample T-Test)
Compares the means of two unrelated groups. This is the most common use case. A marketer comparing conversion rates in Group A (old landing page) vs Group B (new design) uses an independent samples t-test. The test assumes the two samples are drawn independently and that population variances are either equal (Student's t) or unequal (Welch's t). Welch's version is now recommended by default because it performs well even when variances differ. The complete methodology is covered in the two-sample t-test guide.
Sₚ = pooled standard deviation
df = n₁ + n₂ − 2
Sₚ² = [(n₁−1)s₁² + (n₂−1)s₂²] / (n₁+n₂−2)
Paired Samples T-Test
Compares two related measurements from the same subjects — for example, blood pressure before and after taking a medication, or test scores before and after a training program. Because the same people appear in both conditions, the two measurements are correlated, and the test analyzes the difference scores rather than the raw values. This makes it more sensitive than an independent t-test for repeated-measures designs. See the paired samples t-test guide for worked examples.
d̄ = mean of difference scores
Sᵈ = SD of difference scores
n = number of pairs
df = n − 1
What Is ANOVA (Analysis of Variance)?
Ronald Fisher introduced ANOVA in the 1920s while working at Rothamsted Experimental Station, where agricultural researchers needed to compare the effects of multiple fertilizer treatments simultaneously. The key insight was that comparing means is really about comparing variances: if the group means truly differ, the spread between them will be larger than the spread within each group.
How ANOVA Partitions Variance
ANOVA breaks total variability (SS_total) into two pieces:
Variance Partitioning in One-Way ANOVA
| Source | Sum of Squares (SS) | Degrees of Freedom | Mean Square (MS) | F-ratio |
|---|---|---|---|---|
| Between groups | SS_B = Σnⱼ(x̄ⱼ − x̄)² | k − 1 | MS_B = SS_B / (k−1) | F = MS_B / MS_W |
| Within groups | SS_W = Σ(xᵢⱼ − x̄ⱼ)² | N − k | MS_W = SS_W / (N−k) | — |
| Total | SS_T = SS_B + SS_W | N − 1 | — | — |
k = number of groups, N = total observations, x̄ⱼ = group mean, x̄ = grand mean
k = number of groups
N = total sample size
MS_B = between-group mean square
MS_W = within-group mean square (error)
A large F-ratio means the variance between groups is large relative to the variance within groups — evidence that at least one group mean differs from the others. The F-statistic is compared against the F-distribution table at the chosen significance level, with df₁ = k − 1 and df₂ = N − k degrees of freedom.
Types of ANOVA
The ANOVA family covers several research designs:
One-Way ANOVA
One independent variable (factor), three or more levels. Example: comparing test scores across three teaching methods.
Two-Way ANOVA
Two independent variables and their interaction. Example: testing both teaching method and class size, and whether they interact.
Repeated Measures ANOVA
The same subjects are measured across three or more time points or conditions. Example: pain scores at weeks 1, 4, and 8 in a clinical trial.
MANOVA
Multiple dependent variables tested simultaneously. Extends ANOVA to multivariate outcomes, protecting against inflated error across outcomes.
Key Differences Between T-Test and ANOVA
The single clearest dividing line is the number of groups. Beyond that, the tests differ in their test statistics, distributions, and what they tell you after the result is significant:
| Characteristic | T-Test | One-Way ANOVA |
|---|---|---|
| Purpose | Compare 2 group means | Compare 3+ group means |
| Test statistic | t = (x̄₁ − x̄₂) / SE | F = MS_B / MS_W |
| Reference distribution | t-distribution (df = n₁+n₂−2) | F-distribution (df₁ = k−1, df₂ = N−k) |
| Null hypothesis | μ₁ = μ₂ | μ₁ = μ₂ = μ₃ = … = μₖ |
| Alternative hypothesis | μ₁ ≠ μ₂ (or μ₁ > μ₂, μ₁ < μ₂) | At least one μᵢ differs (non-directional only) |
| One-tailed option? | Yes | No |
| Tells you which groups differ? | Yes (only two, so obvious) | No — requires post-hoc tests |
| Type I error control | α per comparison | α per omnibus test |
| Effect size | Cohen's d | Eta squared (η²), partial η², omega squared (ω²) |
| Software output | t-statistic + p-value + CI | ANOVA table + F + p-value |
The Mathematical Relationship: F = t²
When applied to exactly two groups, ANOVA and the independent samples t-test give identical p-values. The F-statistic equals the square of the t-statistic: F = t². This means ANOVA is a generalization of the t-test, not a replacement. For two groups, both are correct — but the t-test is simpler and allows one-tailed tests, which is why it remains the standard choice when k = 2.
With 3 groups you need 3 pairwise t-tests. At α = 0.05, the probability of at least one false positive is 1 − 0.95³ ≈ 14.3%. With 4 groups (6 tests) it rises to 26.5%, and with 5 groups (10 tests) to 40.1%. ANOVA performs a single omnibus test that keeps the overall Type I error rate at α.
Shared Assumptions and Where They Differ
Both tests are parametric and rest on the same core assumptions. Violating them affects the reliability of p-values and confidence intervals. The statistical assumptions guide covers diagnostic methods in detail.
| Assumption | T-Test | ANOVA | How to Check |
|---|---|---|---|
| Normality | Required (robust for n > 30) | Required (robust for balanced n) | Shapiro-Wilk test, Q-Q plots |
| Homogeneity of variance | Required (Welch's t relaxes this) | Required (Welch's ANOVA relaxes this) | Levene's test, Bartlett's test |
| Independence | Required for independent t-test; not for paired | Required for one-way; not for repeated measures | Study design review |
| Continuous outcome | Required | Required | Variable type check |
| Random sampling | Required | Required | Study protocol review |
| No significant outliers | Recommended | Recommended | Boxplot, Z-score, IQR rule |
If normality fails: use the Mann-Whitney U test (2 groups) or Kruskal-Wallis test (3+ groups). If variances are unequal with 2 groups: use Welch's t-test. If variances are unequal with 3+ groups: use Welch's ANOVA. If data are paired but non-normal: use the Wilcoxon signed-rank test.
You can check variance homogeneity using Levene's test. Normality is assessed using Shapiro-Wilk for small samples or Q-Q plots. The normality tests guide walks through both in detail.
Decision Guide: Which Test Do You Need?
The choice between a t-test and ANOVA depends on three things: how many groups you are comparing, whether those groups are independent or related, and whether the parametric assumptions hold. The flowchart below resolves the most common scenarios.
Statistical Test Decision Flowchart
Use the interactive tool below to get a recommendation based on your specific situation:
🔍 Interactive Statistical Test Selector
Worked Examples
Example 1 — Independent Samples T-Test
Problem: A researcher tests whether two teaching methods produce different exam scores. Group A (n = 25, traditional): x̄₁ = 72, s₁ = 8. Group B (n = 25, new method): x̄₂ = 78, s₂ = 9. Assume equal variances. Test at α = 0.05 (two-tailed).
df = n₁ + n₂ − 2 = 48
Sₚ² = pooled variance
State hypotheses: H₀: μ₁ = μ₂ (no difference between methods) | H₁: μ₁ ≠ μ₂ (two-tailed)
Significance level: α = 0.05. With df = 48, the critical value from the t-distribution table is t* ≈ ±2.011.
Check assumptions: Two independent groups, continuous outcome, assume both populations are approximately normal, variances not significantly different (Levene's test p = 0.62 > 0.05). Proceed with pooled t-test.
Calculate pooled variance:
Sₚ² = [(25−1)(8²) + (25−1)(9²)] / (25+25−2)
= [(24)(64) + (24)(81)] / 48
= [1,536 + 1,944] / 48 = 3,480 / 48 = 72.5
Sₚ = √72.5 = 8.515
Calculate test statistic:
SE = 8.515 × √(1/25 + 1/25) = 8.515 × √(0.08) = 8.515 × 0.283 = 2.410
t = (72 − 78) / 2.410 = −6 / 2.410 = t = −2.49
Decision: |t| = 2.49 > t* = 2.011 → p ≈ 0.016 < 0.05 → Reject H₀
✅ Conclusion: At α = 0.05 there is sufficient evidence that the two teaching methods produce different mean exam scores (t(48) = −2.49, p = 0.016). The new method outperformed the traditional method by 6 points on average.
Example 2 — One-Way ANOVA
Problem: Now add a third group — Group C (online instruction): n = 25, x̄₃ = 75, s₃ = 10. Test whether all three teaching methods produce the same mean scores at α = 0.05. k = 3 groups, N = 75 total observations.
df₁ = k − 1 = 2
df₂ = N − k = 72
Hypotheses: H₀: μ_A = μ_B = μ_C (all three methods produce the same mean) | H₁: At least one group mean differs
Significance level: α = 0.05. Critical value: F(2, 72) ≈ 3.12 from the F-table.
Grand mean: x̄ = (25×72 + 25×78 + 25×75) / 75 = (1,800 + 1,950 + 1,875) / 75 = 5,625 / 75 = 75.0
Between-group SS:
SS_B = 25(72−75)² + 25(78−75)² + 25(75−75)²
= 25(9) + 25(9) + 25(0) = 225 + 225 + 0 = 450
MS_B = 450 / (3−1) = 450 / 2 = 225
Within-group SS: Using s² for each group:
SS_W = (25−1)(64) + (25−1)(81) + (25−1)(100)
= 1,536 + 1,944 + 2,400 = 5,880
MS_W = 5,880 / (75−3) = 5,880 / 72 = 81.67
F-statistic: F = MS_B / MS_W = 225 / 81.67 = F = 2.755
⚠️ Conclusion: F(2, 72) = 2.755 < F_critical = 3.12 → p ≈ 0.071 → Fail to Reject H₀. At α = 0.05, there is insufficient evidence that the three teaching methods produce different mean exam scores. Note: the pairwise t-test earlier showed Group A vs B was significant — but once we account for all three groups simultaneously, the omnibus test does not reach significance. This illustrates the importance of using ANOVA when comparing three or more groups rather than cherry-picking pairs.
Post-Hoc Tests: What to Do After a Significant ANOVA
A significant ANOVA F-test tells you that at least one group mean differs, but not which ones. Post-hoc tests (also called multiple comparison procedures) address that question while controlling the overall Type I error rate. Three methods are most commonly used:
| Post-Hoc Test | Best For | Error Control | Internal Link |
|---|---|---|---|
| Tukey HSD | All pairwise comparisons, equal group sizes | Familywise error rate (FWER) | Tukey HSD Guide |
| Bonferroni Correction | Pre-planned comparisons, smaller number of tests | Divides α by number of tests | Bonferroni Guide |
| Scheffé's Method | All possible contrasts, unequal group sizes | Most conservative FWER | Scheffé Table |
| LSD (Fisher) | Exploratory; only valid after significant F | Per-comparison α (liberal) | — |
| Dunnett's Test | Comparing each group to a single control | FWER for control comparisons only | Dunnett's Table |
Tukey HSD is the default recommendation for most situations where group sizes are equal. When groups differ in size or the number of comparisons is small and pre-specified, the Bonferroni correction is a straightforward alternative. See the Tukey's Q table for critical values.
Real-World Applications
Both tests appear across virtually every field that uses data. The pattern is always the same: the number of groups determines the test.
Healthcare / Clinical Research
T-test: Does the new drug lower blood pressure more than a placebo? ANOVA: Which of four dosages (0mg, 5mg, 10mg, 20mg) produces the greatest reduction?
A/B Testing & Marketing
T-test: Does button color A or B produce more clicks? ANOVA: Which of five ad creatives generates the highest conversion rate? See the A/B test calculator.
Education Research
T-test: Do students in bilingual classes score higher than those in monolingual classes? ANOVA: Do test scores differ across four classroom sizes?
Manufacturing / Quality Control
T-test: Does Machine A produce parts with a different mean diameter than Machine B? ANOVA: Which of three production lines has the lowest defect rate?
Agriculture / Ecology
Fisher developed ANOVA specifically for agriculture. T-test: Do two fertilizer formulations produce different crop yields? ANOVA: Which of five fertilizer brands maximizes yield?
Finance / Business Analytics
T-test: Did average revenue change significantly between Q1 and Q2? ANOVA: Do sales figures differ significantly across three regional offices?
Nonparametric Alternatives When Assumptions Fail
When your data are ordinal, heavily skewed, or come from small samples where normality cannot be assumed, parametric tests are unreliable. Use these distribution-free alternatives:
| Situation | Parametric Test | Nonparametric Alternative |
|---|---|---|
| 2 independent groups, non-normal | Independent t-test | Mann-Whitney U test |
| 2 related groups, non-normal | Paired t-test | Wilcoxon signed-rank test |
| 3+ independent groups, non-normal | One-way ANOVA | Kruskal-Wallis test |
| 3+ related groups, non-normal | Repeated measures ANOVA | Friedman test |
These tests rank the raw data rather than using the actual values, which makes them robust to outliers and non-normality. Their disadvantage is lower statistical power than parametric tests when the parametric assumptions are actually met.
Effect Size: Beyond Statistical Significance
Statistical significance tells you whether a difference is real. Effect size tells you how large that difference is in practical terms. A result can be highly significant (p < 0.001) with a tiny effect size that matters very little in practice — especially common in large samples. The effect size guide covers all major measures.
| Test | Effect Size Measure | Formula | Small | Medium | Large |
|---|---|---|---|---|---|
| T-test (2 groups) | Cohen's d | d = (x̄₁ − x̄₂) / Sₚ | 0.2 | 0.5 | 0.8 |
| ANOVA | Eta squared (η²) | η² = SS_B / SS_T | 0.01 | 0.06 | 0.14 |
| ANOVA | Omega squared (ω²) | ω² = (SS_B − (k−1)MS_W) / (SS_T + MS_W) | 0.01 | 0.06 | 0.14 |
| T-test / ANOVA | Power (1−β) | Use sample size calculator | Convention: aim for 0.80 | ||
Use the effect size calculator and sample size calculator to plan your study with adequate power before collecting data. Cohen's d for the t-test worked example above is d = 6 / 8.515 = 0.70, which falls just below the "large" threshold — a practically meaningful difference.
Common Misconceptions
| Misconception | Why It's Wrong | The Reality |
|---|---|---|
| "I can just run 3 t-tests instead of ANOVA for 3 groups." | Inflates Type I error to ~14.3% | Use one-way ANOVA for a single controlled test at α |
| "ANOVA tells you which groups are different." | Omnibus F only says at least one differs | Significant ANOVA requires post-hoc tests (Tukey, Bonferroni) to find which pairs differ |
| "T-test can only be used for small samples." | Sample size determines df, not applicability | T-test is valid at any n; it converges to the z-distribution as n increases |
| "A small p-value means the effect is large." | P-value reflects sample size and effect size combined | Always report effect size (Cohen's d, η²) alongside p-values |
| "ANOVA is more accurate than a t-test for 2 groups." | F = t² when k = 2; they give identical results | For 2 groups, both tests are equivalent; t-test is preferred for directional hypotheses |
| "Failing to reject H₀ proves the groups are equal." | Absence of evidence is not evidence of absence | "Fail to reject H₀" means insufficient evidence — not confirmed equality |
T-Test vs ANOVA Cheat Sheet
- 1 sample vs. known value → One-Sample T-Test (e.g., does our batch mean equal 500g?)
- 2 independent groups → Independent T-Test (e.g., Group A vs Group B exam scores)
- 2 related groups (same subjects) → Paired T-Test (e.g., pre-test vs post-test)
- 3+ independent groups, 1 factor → One-Way ANOVA (e.g., 3 teaching methods)
- 3+ groups, 2 factors → Two-Way ANOVA (e.g., method × class size)
- Same subjects, 3+ time points → Repeated Measures ANOVA (e.g., weeks 1, 4, 8)
- Normality fails, 2 groups → Mann-Whitney U or Wilcoxon Signed-Rank
- Normality fails, 3+ groups → Kruskal-Wallis
- After significant ANOVA → Tukey HSD (or Bonferroni if comparisons are pre-planned)
| Test | Statistic | Degrees of Freedom | Effect Size | Nonparametric Equivalent |
|---|---|---|---|---|
| One-Sample T | t = (x̄ − μ) / (s/√n) | n − 1 | Cohen's d | Wilcoxon Signed-Rank |
| Independent T | t = (x̄₁−x̄₂) / (Sₚ√(1/n₁+1/n₂)) | n₁+n₂−2 | Cohen's d | Mann-Whitney U |
| Paired T | t = d̄ / (Sᵈ/√n) | n − 1 | Cohen's d | Wilcoxon Signed-Rank |
| One-Way ANOVA | F = MS_B / MS_W | k−1 and N−k | η² or ω² | Kruskal-Wallis |
| Two-Way ANOVA | F (3 F-ratios: A, B, A×B) | Multiple | Partial η² | Scheirer-Ray-Hare |
| Repeated Measures ANOVA | F = MS_between / MS_error | Multiple | Partial η² | Friedman |
Case Studies
Case Study — Healthcare
Comparing Pain Reduction Across Three Drug Doses
A clinical pharmacologist studies whether low (5mg), medium (10mg), and high (20mg) doses of a new analgesic differ in their pain reduction scores (VAS scale, 0–10) at 2 hours post-administration. With 30 patients per group (N = 90), she chooses one-way ANOVA because there are three independent groups and the outcome is continuous. After finding a significant F(2, 87) = 8.43, p = 0.0004, she runs Tukey HSD and discovers that 20mg differs from both 5mg (p = 0.0003) and 10mg (p = 0.021), but 5mg and 10mg do not differ from each other (p = 0.31). A t-test comparing only two doses would have missed this full picture — and running three separate t-tests would have inflated the Type I error rate. This study design follows the hypothesis testing framework covered on Statistics Fundamentals.
Case Study — Business Analytics
A/B Test: Two Landing Page Variants
An e-commerce analyst wants to know whether a new landing page layout (Version B) generates higher average order values than the existing layout (Version A). She randomly assigns 500 visitors to each version over a two-week period. This is a classic two-group comparison: she uses an independent samples t-test. The t-test result: t(998) = 2.31, p = 0.021 — Version B generates $4.20 more per order on average. Cohen's d = 0.15, a small but commercially meaningful effect at this transaction volume. If she had wanted to test three page variants simultaneously, she would have used one-way ANOVA to avoid the multiple comparisons problem. See the A/B test calculator for the full workflow.
Glossary of Key Terms
| Term | Definition |
|---|---|
| T-statistic (t) | A standardized test statistic measuring how many standard errors a sample mean is from the null hypothesis value |
| F-statistic (F) | Ratio of between-group mean square to within-group mean square; the test statistic for ANOVA |
| Between-group variance (MS_B) | Average squared deviation of each group mean from the grand mean, weighted by group size |
| Within-group variance (MS_W) | Average squared deviation of individual observations from their own group mean (also called error variance) |
| Degrees of freedom (df) | The number of independent pieces of information used to estimate a parameter; affects the shape of t and F distributions |
| P-value | Probability of observing a test statistic as extreme as the calculated one, assuming H₀ is true |
| Alpha (α) | Pre-set significance threshold; if p < α, reject H₀. Conventionally 0.05 |
| Null hypothesis (H₀) | The default claim of no difference or no effect: μ₁ = μ₂ (t-test) or μ₁ = μ₂ = … = μₖ (ANOVA) |
| Familywise error rate | The probability of at least one false positive across a family of hypothesis tests |
| Post-hoc test | A multiple comparison procedure applied after a significant ANOVA to identify which specific group pairs differ |
| Homogeneity of variance | The assumption that all groups share the same population variance (σ₁² = σ₂² = … = σₖ²) |
| Cohen's d | Effect size for t-tests: the difference between means divided by the pooled standard deviation |
| Eta squared (η²) | Effect size for ANOVA: the proportion of total variance explained by the group factor |
| Type I error (α) | Rejecting a true H₀; a false positive |
| Type II error (β) | Failing to reject a false H₀; a false negative |
| Statistical power (1−β) | Probability of correctly rejecting a false H₀; ideally ≥ 0.80 |
Frequently Asked Questions
t.test(), aov()), Python/SciPy (scipy.stats.ttest_ind(), scipy.stats.f_oneway()), SPSS (Analyze → Compare Means), Excel (Data Analysis Toolpak), Stata, SAS, JMP, and Minitab. For quick calculations, use the t-test calculator and ANOVA calculator on Statistics Fundamentals.