- A test statistic is a function that takes all of the data as input and gives you a single number as the output (usually).
- If the null hypothesis is true, then someone can derive the statistical distribution of the (population) test statistic.
- If you get a sample test statistic (the test statistic that's calculated using your data) that, according to this known probability distribution, is deemed not very likely, then you reject the null.
- The p-value is the probability of getting a sample test statistic that is as much or more extreme than the one that you got in real life; the probability is calculated using the known probability distribution of (population) test statistic conditional on the null hypothesis.
- This methodology makes no sense, yet since everyone uses it, you need to understand it anyway.