The gap between those two numbers is one of the most common and most expensive errors in applied statistics, and it has a name: censoring.
This article works through the example end to end. The dataset is small, public, and from an unusual domain, which makes it useful for teaching — there are no confidentiality issues and the mechanism is easy to see.
The Dataset
A community tracking project recorded 94 predictions and what became of each one. Each prediction was assigned exactly one of four states:
| State | Meaning | Count |
|---|---|---|
| Confirmed | The predicted thing happened | 49 |
| Partial | Some of it happened | 1 |
| Contradicted | The opposite happened | 1 |
| Open | Nothing has happened yet, either way | 43 |
The predictions came from a series of 24 newsletters published between March 2025 and July 2026, each hinting at features the developers of a game intended to release. The tracker recorded, for each hint, whether it eventually shipped in a named update. The full dataset and its method are documented in the public bulletin record kept by the Blox Fruits calculator, which is where these figures come from.
The domain is unusual, but the structure is not. Substitute "clinical trial participants" for predictions and "relapse" for confirmed, and you have a survival analysis. Substitute "sales leads" and "closed-won" and you have a pipeline conversion problem. The shape recurs constantly.
The Naive Calculation
The obvious approach:
= 49 / 94
= 52.1%
This is the number most people would report, and it is the number most dashboards would compute. It says roughly half the predictions came true — which sounds like a coin flip, and invites the conclusion that these predictions carry little information.
The arithmetic is correct. The question being answered is not.
What Went Wrong
Look again at the four states. Three of them describe an outcome: the prediction was confirmed, partly confirmed, or contradicted. The fourth — open — describes an absence of outcome. Nothing has happened yet.
The naive calculation places open observations in the denominator, which silently treats them as failures. But an open prediction has not failed. It has not done anything. It is still running.
The subject was observed, the observation was valid, and the event may still happen after the data was collected. Treating censored observations as failures is equivalent to assuming that anything which has not happened yet never will.
In a dataset where censored cases are a large share of the total — 43 of 94, or 46% here — that assumption dominates the result.
The Corrected Calculation
Restrict the denominator to observations that actually reached an outcome:
= 49 + 1 + 1
= 51
success rate among resolved = 49 / 51
= 96.1%
Among predictions that resolved one way or the other, 96% came true. One was contradicted. One was partial.
These two numbers — 52% and 96% — describe the same dataset. They answer different questions:
Answers: "What share of all predictions have been confirmed so far?" Dependent on when you look. This number will change tomorrow without anything about the predictions changing.
Answers: "When a prediction resolves, how often does it resolve in favour?" A property of the predictions themselves. Only this is a success rate in any useful sense.
Confirming That Censoring Is the Explanation
A corrected number is a hypothesis until you test it. If censoring really is driving the gap, then older observations should show higher confirmed rates than newer ones, purely because they have had more time to resolve.
Splitting the dataset by age:
| Group | Observations | Confirmed | Open |
|---|---|---|---|
| Older (issues 1–19, Mar 2025 – early 2026) | 74 | 59.5% | 37.8% |
| Newer (issues 20–24, July 2026) | 20 | 25.0% | 75.0% |
The newer group has a much lower confirmed rate and a much higher open rate. Nothing suggests the newer predictions are worse — they are simply younger. Three quarters of them have not had an opportunity to resolve.
This is worth checking for whenever a rate appears to be declining over time. A sales team whose recent-quarter close rate looks terrible may simply have deals still in the pipeline. A treatment whose recent cohort shows fewer events may just have less follow-up. If your success rate falls as you approach the present, suspect censoring before you suspect decline.
How Precise Is 96%?
A proportion computed from 51 observations carries real uncertainty, and 96% sits close enough to the boundary that the standard normal-approximation interval behaves badly. The Wilson score interval is the appropriate choice for proportions near 0 or 1.
For 49 successes out of 51, a 95% Wilson interval runs approximately 86.8% to 98.9%.
Upper bound: 98.9%
The honest statement is not "96%". It is: among resolved predictions, the confirmed rate is very high — plausibly anywhere from the high 80s to about 99%, based on 51 resolved cases. That interval is wide because the sample is small. Reporting the point estimate alone would overstate what 51 observations can support.
What This Does Not Tell You
Three limitations matter, and stating them is part of doing this correctly:
As time passes, open observations resolve and move into the denominator. If the ones still outstanding are systematically different — harder, more ambitious, less likely to happen — the resolved rate will fall. Censored data assumes the censoring is unrelated to the outcome. That assumption should be stated, not hidden.
Each prediction was judged against what shipped. Judgement calls at the margins affect the counts, and a single dataset maintained by a small group is not the same as an independently audited one.
A 96% resolved-confirmation rate here says nothing about prediction accuracy anywhere else.
The General Lesson
Whenever a rate is computed, ask two questions before believing it:
- What is in the denominator, and has every item in it had the opportunity to produce the outcome?
- Would this number change if I computed it a month later, without anything underlying it changing?
If the answer to the second is yes, you are looking at a progress measure wearing the costume of a rate.
The correction is usually not difficult. Restricting the denominator to resolved cases, as here, is the simplest form of it. For data where timing matters and censoring is heavy, Kaplan-Meier estimation handles it properly by accounting for how long each observation was actually followed.
What is difficult is noticing the problem in the first place. Censored observations do not announce themselves. They sit quietly in the denominator, dragging every rate downward, and they look exactly like failures on a dashboard.
The 52% figure was not a calculation error. It was a question error — and those are much harder to spot.
Dataset: the bulletin hint-tracking record at the Blox Fruits calculator site, covering 24 newsletters published between March 2025 and July 2026, with figures as at September 2026.