p = .05: Why Does it Matter?

A p-value is the probability of seeing results as extreme as yours (or more extreme) if there were actually no real effect at all. In other words, if the null hypothesis were true. It is not, despite extremely common misuse, the probability that your hypothesis is correct, and it's not the probability that your results happened "by chance" in some general sense either.

A small p-value just means: "if nothing real were actually going on, results like this would be pretty unlikely to show up!'

This is also where a research practice called p-hacking comes in — running many different analyses on the same data until something happens to cross the 0.05 line by chance, then reporting only that result. It's a big part of why some fields have pushed toward pre-registering hypotheses and analysis plans before collecting data, so researchers can't go fishing for significance after the fact.

Because a p-value alone doesn't say anything about how large or meaningful an effect is, many researchers now report an effect size alongside it. An effect size is a separate number describing the actual magnitude of the difference, not just whether it's statistically detectable.

Confidence intervals are another common companion statistic, giving a range of plausible values for the true effect rather than a single yes-or-no significance verdict. A result can be statistically significant with a tiny, practically meaningless effect size, or fail to reach significance with a meaningful effect that a small sample size simply wasn't powerful enough to detect — which is exactly why experienced researchers rarely treat a p-value as the whole story.

If your project involves any statistics, you'll likely end up calculating a p-value yourself at some point, whether through a t-test, chi-square test, or something more advanced. Being able to explain what that number actually means instead of just reporting "p ‹ 0.05" and moving on is important if you’re trying to defend your research.