What is the Wald-Wolfowitz Runs Test?

The Wald-Wolfowitz Runs Test is a nonparametric statistical test that determines whether a binary sequence of two types of elements is random. A run is a consecutive subsequence of identical elements. The test counts the total number of runs r and compares it to critical values based on n₁ and n₂ — the counts of each element type. Too few or too many runs both indicate non-randomness.

How do you calculate the number of runs?

Count the number of unbroken sequences of identical elements in your ordered data. For the sequence H H T H T T T H, the runs are: (HH), (T), (H), (TTT), (H) — giving r = 5 runs. Each time the element type changes, a new run begins. n₁ = count of H = 4, n₂ = count of T = 4.

How do you use the Runs Test table?

To use the Runs Test table: (1) Record your binary sequence and count n₁ and n₂. (2) Count the total number of runs r. (3) Locate the row for n₁ and the column for n₂ in the table. (4) Read off r_L (lower critical value) and r_U (upper critical value). (5) Apply the decision rule: if r ≤ r_L or r ≥ r_U, reject H₀ and conclude the sequence is not random at α = 0.05.

What is the decision rule for the Runs Test?

Reject H₀ (randomness) if: r ≤ r_L (too few runs — clustering pattern) OR r ≥ r_U (too many runs — alternating pattern). Fail to reject H₀ if r_L < r < r_U. The test is two-tailed: both extremes indicate non-randomness, just in different directions.

When should you use the Runs Test instead of other tests?

Use the Runs Test when your data form a binary ordered sequence and you want to test whether the order is random. It is appropriate for coin flip sequences, quality control inspection records (pass/fail), medical treatment outcomes (success/failure), financial market direction sequences (up/down), and any other situation where two outcomes appear in a meaningful order. Unlike the chi-square test, the Runs Test preserves order information.

What are the assumptions of the Runs Test?

The Runs Test requires: (1) the data form an ordered sequence, (2) each observation falls into exactly one of two categories, (3) observations are independent of one another, and (4) the sequence is binary — only two possible values. The test makes no assumptions about the underlying population distribution, which is why it is nonparametric.

What does the expected number of runs formula give?

The expected number of runs under H₀ (perfect randomness) is: E(r) = (2·n₁·n₂)/(n₁+n₂) + 1. The variance is: Var(r) = 2·n₁·n₂·(2·n₁·n₂ − n₁ − n₂) / [(n₁+n₂)²·(n₁+n₂−1)]. For large samples (n₁ > 20 or n₂ > 20), the standardized Z statistic Z = (r − E(r)) / √Var(r) approximately follows a standard normal distribution.

Is the Runs Test parametric or nonparametric?

The Runs Test is nonparametric. It makes no assumptions about the underlying probability distribution of the data. This makes it suitable for any binary ordered sequence, regardless of whether the generating process follows a normal or any other specific distribution.

Can the Runs Test detect both clustering and alternating patterns?

Yes. The Runs Test is two-tailed and detects both departures from randomness. Too few runs (r ≤ r_L) suggests clustering — the same elements tend to group together. Too many runs (r ≥ r_U) suggests alternating — elements systematically switch back and forth. Both patterns indicate the sequence is not random.

Runs Test Table: Critical Values, Decision Rules & How to Use It

Runs Test Critical Value Calculator

n₁ (count of type 1)

n₂ (count of type 2)

Observed runs (r)

Lower Critical (r_L)

—

Upper Critical (r_U)

—

What Is the Runs Test Table?

The Runs Test table — also called the Wald-Wolfowitz Runs Test table — gives the critical values needed to test whether a binary sequence is random. The table has two critical values for each combination of n₁ and n₂: a lower critical value r_L and an upper critical value r_U.

A run is an unbroken sequence of identical elements. In the sequence H H H T T H T T T, there are four runs: (HHH), (TT), (H), (TTT). The total number of runs r is what the test compares against the critical values from the table. Both too few runs (clustering) and too many runs (systematic alternating) are evidence against randomness.

Key distinction: r is what you count from your data; r_L and r_U are what you look up. The test is two-tailed — you reject H₀ if r is either too small or too large relative to what random sequences produce.

Runs Test Critical Value Table (α = 0.05)

Select the tab to view lower critical values (r_L) or upper critical values (r_U). Find the row for n₁ and the column for n₂. Click any cell to load it into the calculator above.

Showing: Lower critical values r_L at α = 0.05 (two-tailed). Reject H₀ if r ≤ r_L.

α = 0.05 (two-tailed). n₁ = count of element type 1; n₂ = count of element type 2. Reject H₀ if r ≤ r_L OR r ≥ r_U. Source: Swed, F. S. & Eisenhart, C. (1943). Tables for testing randomness of grouping in a sequence of alternatives. Annals of Mathematical Statistics, 14(1), 66–87.

What Is a Run? Understanding the Core Concept

A run is a maximal sequence of identical consecutive elements. Every time the element type changes, the current run ends and a new one begins.

Example: Counting Runs in a Coin Flip Sequence

Sequence: H H T H T T T H H T

H H T H T T T H H T

Runs: (HH) | (T) | (H) | (TTT) | (HH) | (T) → r = 6 runs. n₁ = 5 (Heads), n₂ = 5 (Tails).

From the table: n₁ = 5, n₂ = 5 → r_L = 2, r_U = 10. Since 2 < 6 < 10, fail to reject H₀. The sequence does not differ significantly from random.

Too Few Runs (Clustering)

HHHHTTTT

r = 2. Elements of the same type cluster together. r ≤ r_L → reject H₀.

Random (Near Expected)

HTHHTHTT

r = 7. Runs are distributed as expected under randomness. Fail to reject H₀.

Too Many Runs (Alternating)

HTHTHTHT

r = 8. Every element alternates. r ≥ r_U → reject H₀.

How the Wald-Wolfowitz Runs Test Works

The Runs Test follows a structured five-step procedure. Each step corresponds to a specific statistical decision.

Step 1 — State Hypotheses

The null hypothesis is H₀: The sequence is random (the order of elements was generated by a random process). The alternative is H₁: The sequence is not random (the order shows systematic patterns). Set α = 0.05 before collecting data. The test is always two-tailed — both too few and too many runs indicate non-randomness.

Step 2 — Classify Elements and Count n₁, n₂

Each observation in your ordered sequence must belong to one of exactly two categories (type 1 and type 2). Count n₁ (total observations of type 1) and n₂ (total observations of type 2). The table covers n₁ and n₂ from 2 to 20.

n = n₁ + n₂ (total observations in the sequence)

Step 3 — Count Runs (r)

Scan the sequence from left to right. r = 1 at the start. Every time the element type changes, add 1 to r. The final r is the total number of runs. For large samples (n₁ > 20 or n₂ > 20), use the Z approximation instead of this table — see the large-sample section below.

Step 4 — Look Up Critical Values

Find the row for n₁ and the column for n₂ in the table above. Read off r_L (lower critical value from the first tab) and r_U (upper critical value from the second tab). If the table does not list your exact n₁/n₂ pair, use the large-sample Z approximation.

Step 5 — Apply the Decision Rule

If r ≤ r_L → Reject H₀ → Too few runs → Clustering pattern detected
If r ≥ r_U → Reject H₀ → Too many runs → Alternating pattern detected
If r_L < r < r_U → Fail to reject H₀ → Sequence is consistent with randomness

Runs Test Formulas

Three formulas govern the Runs Test. The expected value and variance apply to all sample sizes; the Z statistic is used when n₁ or n₂ exceeds 20.

Expected Number of Runs Under H₀

E(r) = (2·n₁·n₂) / (n₁ + n₂) + 1

This gives the average number of runs expected if the sequence were truly random. An observed r close to E(r) supports H₀.

Variance of Runs Under H₀

Var(r) = [2·n₁·n₂ · (2·n₁·n₂ − n₁ − n₂)] / [(n₁ + n₂)² · (n₁ + n₂ − 1)]

Large-Sample Z Statistic (n₁ > 20 or n₂ > 20)

Z = (r − E(r)) / √Var(r)

When sample sizes exceed 20, Z approximately follows a standard normal distribution. Reject H₀ at α = 0.05 (two-tailed) if |Z| > 1.96. Use the Z-table to find p-values.

Worked Example 1: Quality Control Inspection

Scenario: A quality engineer inspects 18 consecutive items from a production line and records each as Pass (P) or Fail (F). The sequence is: P P P F F P P F P P P P F F F P P F. She wants to test at α = 0.05 whether defects occur randomly or cluster together.

Solution — Step by Step

Step	Action	Result
1	State H₀ and H₁	H₀: Defects are random; H₁: Defects are not random; α = 0.05
2	Count n₁ and n₂	n₁ (P) = 12, n₂ (F) = 6
3	Count runs	(PPP)(FF)(PP)(F)(PPPP)(FFF)(PP)(F) → r = 8
4	Look up critical values	n₁=12, n₂=6 → r_L = 4, r_U = 13
5	Apply decision rule	4 < 8 < 13 → Fail to reject H₀

Interpretation

With r = 8 runs, n₁ = 12, n₂ = 6, we fail to reject H₀ at α = 0.05. The distribution of defective items along the production line is consistent with a random process. No evidence of systematic clustering or alternating patterns in this sample. E(r) = (2×12×6)/(12+6) + 1 = 9, so r = 8 is close to what randomness predicts.

Worked Example 2: Stock Market Direction Test

Scenario: A financial analyst records daily closing price movements for 20 trading days: U D D D U D D D U D D D U D D D U D D D (U = price Up, D = price Down). She tests whether the direction of price movement is random.

Solution

Step	Action	Result
1	State H₀ and H₁	H₀: Price movements are random; H₁: Not random; α = 0.05
2	Count n₁ and n₂	n₁ (U) = 4, n₂ (D) = 16
3	Count runs	(U)(DDD)(U)(DDD)(U)(DDD)(U)(DDD) → r = 8
4	Look up critical values	n₁=4, n₂=16 → r_L = 3, r_U = 10
5	Apply decision rule	3 < 8 < 10 → Fail to reject H₀

Interpretation

Despite the visual pattern of one Up day followed by three Down days repeating four times, the Runs Test does not reject randomness at α = 0.05. With n₁ = 4 and n₂ = 16, a very unequal split, the critical region is wider and the test has less power to detect the pattern. Note: E(r) = (2×4×16)/(4+16) + 1 = 7.4, so r = 8 is close to expected.

Large-Sample Runs Test: Z Approximation

When n₁ or n₂ exceeds 20, the exact table no longer applies. Instead, the number of runs r approximately follows a normal distribution, so a Z test is used.

Large-Sample Worked Example

Scenario: A researcher records 50 patient outcomes: n₁ (Improved) = 27, n₂ (Not Improved) = 23, r = 18 runs. Test randomness at α = 0.05.

Calculation	Formula	Value
Expected runs	E(r) = (2×27×23)/(27+23) + 1	25.84
Variance	Var(r) = [2×27×23×(2×27×23−50)] / [50²×49]	11.87
Z statistic	Z = (18 − 25.84) / √11.87	−2.27
Decision	\|Z\| = 2.27 > 1.96	Reject H₀

With Z = −2.27, |Z| > 1.96, we reject H₀ at α = 0.05. The sequence shows significantly fewer runs than expected under randomness — patients with similar outcomes tend to cluster together in time.

Runs Test vs Other Nonparametric Tests

The choice among nonparametric tests depends on what you want to test and what type of data you have. The table below maps each test to its purpose.

Test	Tests For	Data Type	Order Matters?	Related Table
Runs Test	Randomness of sequence	Ordered binary	Yes	This page
Sign Test	Median or paired differences	Paired ordinal	No	Hypothesis Testing
Chi-Square Test	Association between categories	Categorical	No	Chi-Square Table
Binomial Test	Probability of success vs. p₀	Binary, count	No	Binomial Table
Mann-Whitney U	Location difference between two groups	Ordinal/continuous	No	Mann-Whitney Table
Autocorrelation	Serial dependence in time series	Continuous ordered	Yes	Regression

Assumptions of the Runs Test

The Runs Test is nonparametric, but it still requires certain data conditions to be met for valid conclusions.

Binary Observations

Each observation must be one of exactly two types. Continuous data can be converted to binary by splitting at the median (above/below).

Ordered Sequence

The data must form a meaningful sequence where the order of observations matters. Time series, spatial sequences, and inspection records are typical examples.

Independence

Under H₀, each element's type is determined independently of other elements. The test checks whether this independence holds — if it does not, the sequence is non-random.

Sample Size Range

The exact table applies when 2 ≤ n₁ ≤ 20 and 2 ≤ n₂ ≤ 20. For larger samples, use the Z approximation. Very small samples (n₁ or n₂ = 1) cannot be meaningfully tested.

Applied Examples Across Research Fields

The Runs Test applies wherever ordered binary data arise. The following examples show the diversity of contexts where the test gives meaningful answers.

Manufacturing: Statistical Process Control

A quality engineer tracks whether 15 consecutive machine outputs are within spec (Pass) or outside spec (Fail). The Runs Test checks whether failures cluster together, suggesting a systematic process shift, or appear randomly, suggesting isolated variation. Clustering of failures typically indicates a machine calibration problem. See the hypothesis testing framework for the broader context.

Clinical Research: Treatment Outcome Sequences

In a clinical trial, researchers record consecutive patient outcomes as Success (S) or Failure (F). If outcomes cluster — many successes together followed by many failures — this may indicate that patient enrollment is not random or that external conditions are changing over time. The Runs Test detects this without assuming normality. Relevant to the study design considerations for randomization.

Finance: Market Direction Testing

The efficient market hypothesis predicts that daily price movements are random. Researchers apply the Runs Test to sequences of Up (U) and Down (D) days. Significantly too few runs suggests market momentum (trending); significantly too many suggests mean-reversion. The test is a classic tool in financial time series analysis, used before applying regression models.

Psychology: Response Pattern Analysis

In cognitive psychology experiments, the Runs Test checks whether a participant's binary responses (correct/incorrect, yes/no) are random or show systematic patterns like blocks of correct responses followed by blocks of errors. This helps identify learning effects, fatigue, or response strategies that deviate from random guessing.

When to Use the Runs Test: A Decision Guide

Use this guide to decide whether the Runs Test fits your research situation.

Situation	Use Runs Test?	Alternative
Ordered binary sequence, testing randomness	Yes ✓	—
Continuous data; split at median to binarize	Yes ✓	Autocorrelation if continuous preserved
n₁ or n₂ > 20	Z approx.	Use Z = (r − E(r))/√Var(r)
Comparing two group medians (not order)	No ✗	Mann-Whitney U Test
Testing frequency of categories (no order)	No ✗	Chi-Square Test
Testing proportion equal to a known p₀	No ✗	Proportion Z-Test

Common Mistakes When Using the Runs Test Table

These are the errors researchers most often make when applying the Runs Test. Each is easy to avoid once identified.

Confusing n₁ and n₂ with total n

n₁ and n₂ are the counts of each element type, not the total sample size. With 10 Heads and 8 Tails, n₁ = 10, n₂ = 8, and n = 18. Use n₁ and n₂ to look up the table, not n.

Using only one critical value (one-tailed thinking)

The Runs Test is always two-tailed. You need both r_L and r_U. Rejecting H₀ only for r ≤ r_L misses the alternating-pattern case. Always check both bounds.

Ignoring the order of observations

The Runs Test depends entirely on the sequence order. Rearranging the same elements produces a different r and a different conclusion. The test is only meaningful when the original data order reflects a real temporal or spatial process.

Applying the table when n₁ or n₂ > 20

This table is valid for n₁ ≤ 20 and n₂ ≤ 20. For larger samples, the exact table does not apply. Use the Z approximation formula: Z = (r − E(r)) / √Var(r) and compare |Z| to 1.96.

Symbol and Concept Glossary

Every symbol in the Runs Test, defined with its role in the test and the most common way it gets misapplied.

Symbol	Name	Definition	Common Error
r	Observed runs	Count of unbroken sequences of identical elements	Counting elements instead of runs
n₁	Count of type 1	Total observations of the first element type	Using n (total) instead
n₂	Count of type 2	Total observations of the second element type	Using n (total) instead
r_L	Lower critical value	Minimum runs for non-rejection of H₀; reject if r ≤ r_L	Ignoring the upper bound
r_U	Upper critical value	Maximum runs for non-rejection of H₀; reject if r ≥ r_U	Treating the test as one-tailed
E(r)	Expected runs	(2·n₁·n₂)/(n₁+n₂) + 1; average runs under H₀	Forgetting +1 in the formula
α	Significance level	Probability of Type I error; this table uses α = 0.05	Setting α after seeing the data
Z	Standardized statistic	(r − E(r))/√Var(r); used when n₁ or n₂ > 20	Using Z table when n₁, n₂ ≤ 20

Quick Lookup: Common Critical Value Pairs (α = 0.05)

These are the most commonly referenced critical value pairs for balanced samples where n₁ = n₂.

n₁ = n₂	r_L (lower)	r_U (upper)	E(r)	Reject H₀ if
5	2	10	6.0	r ≤ 2 or r ≥ 10
8	5	13	9.0	r ≤ 5 or r ≥ 13
10	6	16	11.0	r ≤ 6 or r ≥ 16
12	8	18	13.0	r ≤ 8 or r ≥ 18
15	10	22	16.0	r ≤ 10 or r ≥ 22
20	13	29	21.0	r ≤ 13 or r ≥ 29

Runs Test in Statistical Software

All major statistical software packages include the Runs Test. The implementations below match the table-based approach used on this page.

R

# Install if needed
library(tseries)
# Define sequence
x <- c("H","H","T","H","T","T","H")
# Run the test
runs.test(as.factor(x))

The tseries package returns the test statistic, p-value, and runs count. Source: CRAN tseries

Python (SciPy)

from scipy.stats import runstest_1samp
import numpy as np
# Binary sequence as 0/1
x = np.array([1,1,0,1,0,0,1])
stat, p = runstest_1samp(x)
print(f"Z={stat:.3f}, p={p:.4f}")

Uses the Z approximation. Source: SciPy docs

SPSS

/* SPSS Syntax */
NPAR TESTS
/RUNS(MEAN) = sequence_var
/MISSING ANALYSIS.

In SPSS: Analyze → Nonparametric Tests → Legacy Dialogs → Runs. Cut point options: mean, median, or custom value.

References and Further Reading

Swed, F. S., & Eisenhart, C. (1943). Tables for testing randomness of grouping in a sequence of alternatives. Annals of Mathematical Statistics, 14(1), 66–87. The original source of the Runs Test critical value tables used in this reference.

Wald, A., & Wolfowitz, J. (1940). On a test whether two samples are from the same population. Annals of Mathematical Statistics, 11(2), 147–162. The foundational paper introducing the Runs Test as a nonparametric test of randomness and the two-sample problem.

NIST/SEMATECH e-Handbook of Statistical Methods (2013). Section 1.3.5.14: Runs Test for Detecting Non-Randomness. National Institute of Standards and Technology. itl.nist.gov

Conover, W. J. (1999). Practical Nonparametric Statistics (3rd ed.). Wiley. Chapter 3 covers the Runs Test, exact critical values, and applications across quality control and time series analysis. Standard graduate-level nonparametric statistics text.

Penn State STAT 415: Introduction to Mathematical Statistics. Wald-Wolfowitz Runs Test lecture notes. Pennsylvania State University. online.stat.psu.edu/stat415/

Related Statistical Tables & Resources

χ²

Chi-Square Table Test association between categories

Mann-Whitney U Table Compare two independent groups

Wilcoxon Signed-Rank Table Test paired differences nonparametrically

Binomial Table Exact binomial probabilities

Z-Table Standard normal critical values

🏠

Statistics Fundamentals Tables, calculators & complete guides

How to Use This Table

1Label observations as type 1 or type 2

2Count n₁ and n₂ separately

3Count runs r in sequence

4Find r_L and r_U from table

5Reject H₀ if r ≤ r_L or r ≥ r_U

Quick Reference (α = 0.05)

n₁=n₂=5 r_L=2, r_U=10

n₁=n₂=8 r_L=5, r_U=13

n₁=n₂=10 r_L=6, r_U=16

n₁=n₂=15 r_L=10, r_U=22

n₁=n₂=20 r_L=13, r_U=29

Key Formulas

E(r) = (2n₁n₂)/(n₁+n₂) + 1
Z = (r − E(r)) / √Var(r)
Reject if r ≤ r_L or r ≥ r_U

Statistics Fundamentals

Tables, calculators & complete guides

More Tables

→ Chi-Square Table → Mann-Whitney U Table → Wilcoxon Signed-Rank Table → Z-Table → T-Distribution Table → Binomial Distribution Table

Understanding What the Runs Test Table Tells You

Why Both Critical Values Are Needed

The Runs Test has two rejection regions because non-randomness can go in two directions. A sequence with very few runs shows clustering — elements of the same type group together. A sequence with very many runs shows systematic alternation. Both patterns are statistically unusual under randomness, and the table catches both.

Why Unequal n₁ and n₂ Affect the Test

When n₁ and n₂ are very different, the maximum possible number of runs is limited by the smaller group. A sequence with n₁ = 2 and n₂ = 18 cannot have more than 5 runs regardless of the arrangement. The table accounts for this by providing tighter critical bounds when one group is much smaller. Always look up the specific (n₁, n₂) pair, not a symmetric approximation.

Runs Test vs Autocorrelation for Continuous Data

For continuous time series, autocorrelation analysis preserves the magnitude of each observation. The Runs Test only uses the sign of each observation relative to the median — it loses information but gains robustness. When the distribution is non-normal or the data are ordinal, the Runs Test is often preferred. Both approaches are part of broader hypothesis testing practice.