intuition/logic behind comparing p-value and significance level [duplicate]

I'm new on this forum and I hope that this is not already asked. My question is this:

I know that the p-value is defined as

$$p=P($$obtaining sample as or more contradictory to $$H_0$$ then the one obtained$$|H_0$$is true$$)$$

and the significance level is

$$\alpha=P($$reject $$H_0 | H_0$$is true$$)$$.

What is the logic/intuition behind rejecting the null-hypothesis when $$p < \alpha$$?

Kind regards,

marked as duplicate by whuber♦Aug 15 at 18:52

Suppose I have the following $$n = 20$$ observations sampled at random, as listed and summarized below:

x
[1] 37.0 38.4 46.2 57.1 40.2 39.0 54.7 54.0 52.5 37.1
[11] 49.6 45.7 43.6 41.1 49.7 56.3 41.8 53.8 48.1 38.3
summary(x)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
37.00   39.90   45.95   46.21   52.83   57.10
sd(x)
[1] 6.919454


I suppose that the population from which these data were randomly sampled is normal, and I want to test $$H_0: \mu = 50$$ against $$H_a: \mu \ne 50,$$ where $$\mu$$ is the population mean. The sample mean is $$\bar X = 46.2,$$ which is somewhat below $$50.$$ I use a t test to find out if $$\bar X$$ is 'significantly' different from $$50.$$

t.test(x, mu=50)

One Sample t-test

data:  x
t = -2.4495, df = 19, p-value = 0.02417
alternative hypothesis: true mean is not equal to 50
95 percent confidence interval:
42.9716 49.4484
sample estimates:
mean of x
46.21


I want to do this test at the 5% level of significance. If $$\mu$$ really is $$50,$$ I want the probability of rejecting $$H_0$$ in favor of the alternative $$H_a$$ (a mistake) to be small: below 5%.

The t statistic, $$T = -2.4495$$ above, is distributed according to Student's t distribution with degrees of freedom $$\nu = n - 1$$ $$= 20 - 1 = 19.$$ That distribution puts 95% of its probability between $$\pm 2.228$$ (vertical red dotted lines in the plot below). These are the upper and lower 'critical values'.

The R code below finds the upper critical value. You can also find this critical value in printed tables of Student's t distribution.

qt(.975, 10)
[1] 2.228139


The observed value of the t statistic, $$T = -2.4495,$$ shown in the plot as a vertical blue line, lies beyond the lower critical value. So we say that the null hypothesis $$H_0$$ is rejected at the 5% level of significance.

A second way to see that we reject $$H_0$$ at the 5% level is shown in the output from the t test above. If $$H_0$$ is true, then the probability $$P(|T| \ge -2.4495)= P(T \le -2.4495) + P(T \ge 2.4495) = 0.2417.$$ If $$H_0$$ is true, this is the probability that the $$T$$-statistic lies farther from 0 (in either direction) than the observed value $$T = -2.4495.$$ We know to reject $$H_0$$ at the 5% level of significance because the P-value is smaller than 0.05.

2 * pt(-2.4495, 19)
[1] 0.02417408


Generally speaking, there is not sufficient information in printed tables of Student's t distribution to find an exact P-value. So you will ordinarily have to rely on computer output (or results from a statistical calculator) to compute P-values.

Using p-values has one advantage. If I want to test the null hypothesis at the 1% level, I know that I cannot reject $$H_0$$ at the 1% level because the P-value exceeds 0.01. (So it is not necessary to 'tell' a computer program what your level of significance is. Once you get the P-value you can test at any desired level of significance.)

Note: I was not surprised that the t test rejected the null hypothesis because I simulated the 20 observations in R, sampling from a normal distribution with $$\mu = 47.$$

set.seed(815)
x = round(rnorm(20, 47, 7),1)