Why Bayes fits everyday classification
Most real classification starts with partial information. A spam filter sees a few words. A clinician sees a few symptoms. A homeowner sees antennae and wings. No single observation proves anything, but each one shifts the odds.
The Bayesian approach separates two quantities people tend to blur together. The prior is how common each class is before you look closely. The likelihood is how often the observed feature shows up within each class. Multiply them, normalize, and you get the posterior: the updated probability you actually act on.
Keeping the two apart matters because people routinely skip the prior, a mistake called base-rate neglect.
Setting priors from base rates that keep shifting
Textbooks often treat a prior as a fixed number. In practice, the base rates behind many classification problems move, and pest populations show this well. According to News from the state's land-grant university, Florida has 20 recorded termite species, 14 native and six invasive, and only a few are treated as pests.
| Measure | Figure |
|---|---|
| Termite species recorded in Florida | 20 |
| Native termite species | 14 |
| Invasive termite species | 6 |
| Asian subterranean termite flight peak | March to April |
| Formosan subterranean termite flight peak | April to May |
| West Indian drywood termite flight peak | May to June |
For a statistician, the flight windows are the interesting column. They tie the prior to the calendar. Swarming termites concentrate in spring and early summer, so a winged insect indoors in South Florida in early April starts with a different prior than the same insect in November. Invasive species spreading into new areas also push the regional prior upward over time, which means a prior built from older sightings data can go stale.
So write down where your prior came from and when. Revisit it when the season or the data source changes.
Updating with observed features, step by step
Suppose that among winged insects found indoors in spring, 60% are termites and 40% are carpenter ants. These numbers are illustrative, chosen for this tutorial, not taken from a survey. The process:
- Set the prior: P(termite) = 0.60 and P(ant) = 0.40.
- Assign likelihoods for straight antennae: P(straight | termite) = 0.90 and P(straight | ant) = 0.10.
- Multiply each prior by its likelihood and add the products to get the evidence term.
- Divide each product by that sum to get the posterior, then repeat with the next feature.
For straight antennae, the termite product is 0.60 × 0.90 = 0.54 and the ant product is 0.40 × 0.10 = 0.04. The evidence term is 0.58, so P(termite | straight antennae) = 0.54 / 0.58, or about 0.931. One feature took you from 60% to 93%.
Now add wing length. Say P(equal wings | termite) = 0.85 and P(equal wings | ant) = 0.20; the ant figure is higher than you might expect because a blurry photo can make ant wings look closer in size than they are. If you assume the features are conditionally independent (the naive Bayes shortcut), multiply again: 0.54 × 0.85 = 0.459 for termites and 0.04 × 0.20 = 0.008 for ants. The sum is 0.467. The termite posterior is 0.459 / 0.467, about 0.983.
To see how much the prior matters, rerun the first update with P(termite) = 0.30, a reasonable winter value. The products become 0.27 and 0.07, and the posterior after straight antennae falls to about 0.794. Same insect, same feature, a clearly different answer, and the only thing that changed was the base rate.
Prior × Likelihood, normalized across all classes, gives the posterior. Repeat for each new piece of evidence.
Checking the posterior with a tree
Most arithmetic errors in Bayesian problems come from leaving a branch out of the denominator. Drawing a probability tree catches this quickly. The first split is the class (termite 0.60, ant 0.40). The second is the feature (straight or elbowed antennae). Every endpoint carries the product of its branches, and the posterior is the termite-and-straight endpoint divided by the total of all endpoints showing straight antennae.
The tree also makes the independence assumption visible. If antennae shape and wing appearance tend to be misread together, say because both depend on photo quality, the naive multiplication overstates your confidence. Treat 0.98 as an upper bound, not a final answer.
The features themselves also arrive differently now. More identifications start from a phone photo or an image-recognition app than from a specimen under a hand lens, and that raises the chance of misreading a trait. In statistical terms, it narrows the gap between the two likelihoods and weakens each update. A good model uses likelihoods that match how the observation was actually made, not textbook values.
For practice, enter the numbers above into the site's Bayes and probability tree tools and change one input at a time: first the prior, then each likelihood. Track which change moves the posterior most. Drop the prior from 0.60 to 0.30 and the one-feature answer falls from 0.931 to 0.794, a bigger swing than most likelihood tweaks produce. That is the number to check first, and the one people most often forget to update.
- Prior: how common each class is before observing anything
- Likelihood: how often a feature appears within each class
- Posterior: updated probability after multiplying and normalizing
- Naive Bayes: assumes features are conditionally independent — useful shortcut, but overstates confidence when features correlate
- Base-rate neglect: the common error of skipping or underweighting the prior
- Probability tree: the fastest way to catch a missing denominator branch