The p-value sentence I kept getting wrong, until I had to say it out loud

What is actually wrong with it

The sentence has the condition the wrong way round. A p-value is calculated by assuming the null hypothesis is true and then asking: if that were the case, how often would I see data at least this extreme? It is a probability about the data, given the null. It is not a probability about the null, given the data. A hypothesis test only ever computes the first one.

A small p-value says that data like yours would be unusual if the null held. Either something unusual happened, or the null is not a good description of what produced your data. That is a reason to doubt the null. It is not the chance that the null is false, and one minus the p-value is not the chance that your alternative is true. Plain hypothesis testing does not hand out probabilities for hypotheses at all; that is a different framework.

I could recite all of that. I still kept writing the wrong sentence.

Why saying it beats writing it

When you write, you can leave gaps and your eye fills them in. When you say it to someone who will answer back, the gaps are audible. The "if the null hypothesis is true" part either comes out of your mouth or it does not. That is the old explain-it-as-if-teaching trick: say the idea in your own words, get told where it broke, say it again. The hard part for a first-year is finding the someone. My lab partner is as new to this as I am. My tutor has one office hour a week and a queue outside it.

The session

I ended up doing it with the free ai voice chat on rewind.ai, because it answers out loud and did not ask me to make an account first. The whole page is built around one microphone: you talk, you stop, it talks. In the library I used the text field under it instead, and I swapped the default voice for a calmer one in the settings.

For the first go I typed, because I wanted the exact sentence from my problem sheet. I asked it to tell me plainly what I got wrong and then ask one follow-up question, in spoken sentences with no lists. Then I gave it the sentence.

The reply appeared on screen and was spoken at the same time; I could have cut it off, and did not. In short: that is not quite right; a p-value is the probability of observing your data, or something more extreme, if the null hypothesis is true; it is not the probability that the null hypothesis is true. Then the follow-up: can you explain why a small p-value might make us doubt the null?

My sentence, the correction, and the question it handed back.

That question is the useful bit. Being asked to carry the corrected idea one step further is where it stuck. I said my answer out loud before I typed a word of it: if the null were true, data this extreme would be rare, so seeing it is a reason to think the null is a poor fit. Then I said the corrected definition back, twice. Hearing a correction and producing it are different things.

Where else I tried the same sentence

ChatGPT, without logging in, answers typed questions. It told me, correctly, that a p-value is the probability of results at least as extreme as the observed ones, assuming the null is true. Its microphone turns speech into typing and nothing more, so the answer sat on the screen in silence. For the saying-it-out-loud part it is a notes app with opinions.

Gemini is the same shape. It answered correctly too, as text. Then a strip slid in suggesting I sign in and link it to my Google apps. Talking back is not what you get at the door.

Khanmigo, Khan Academy's tutor, is the one people on my course mention most. Its learners page says you can speak to it and it will read its responses out, and that it is built to never give you the answer, which is the opposite of what I wanted here. The Get Khanmigo button goes to a Khan Academy sign-up page, with a monthly fee behind it. I did not go further.

Then there is the study group. Four of us, a whiteboard, Thursday afternoons. Useless for this: when I said the wrong sentence out loud in week four nobody corrected me. Two of them believed it too, and the one who did not was too polite to say so. A tool has no manners, which turns out to be the feature.

Three more sentences worth saying out loud

The ones my study group gets wrong, run through the same loop.

The ninety-five percent belongs to the method: repeat the sampling many times, build an interval each time, and about ninety-five in a hundred of those intervals would cover the true mean. Any single interval either covers it or does not.

The standard deviation describes how spread out the individual values are. The standard error of the mean describes how much the sample mean would wander from one sample to the next; it is the standard deviation divided by the square root of the sample size, so it shrinks as the sample grows while the spread of the data does not.

A large p-value means the data are compatible with the null. Compatible is not confirmed. You failed to reject it, which is a statement about your evidence, not about the world.

How I set it up now

Set the rules before you say the idea: I am going to explain a concept from my stats course, tell me plainly what I got wrong, then ask one follow-up question, plain sentences, no lists. That last part keeps it sounding like a conversation rather than a lecture.

Give it your actual sentence, the one from your homework, not a cleaned-up version. Answer the follow-up before you look anything up. Then say the corrected sentence back, out loud, full stop at the end.

And check what it told you against the lecture notes. Mine matched the textbook, but any AI can be wrong in a confident voice, and the mark in the margin is the one that counts.