# Why is $\alpha$ called the significance level if it is inversely related to the significance of a given rejection or non-rejection of $H_0$

I know what the significance level is and how hypothesis testing works, so this is primarily a terminological question.

In hypothesis testing, the so-called "significance level" $\alpha$ equals the probability of falsely rejecting a true null hypothesis. If this value is particularly low for a given test for which the null hypothesis has been rejected, this means that the result is very unlikely to have been obtained by chance alone. This yields the very counterintuitive result that when considering a given hypothesis test and the associated decision of rejection or non-rejection of the null hypothesis, a lower significance level means a more highly significant result. The lower the significance level, the higher the significance of the result...

Intuitively, one would expect "significance level" to mean something similar to "confidence level." Unfortunately, it means the opposite. I find that this confuses undergraduates quite a bit and it would be helpful to have a good explanation for them as to how naming $\alpha$ the significance level makes sense.

So, what are the historical, statistical, or mathematical reasons that the significance level inversely related to the confidence level?

It may help to recall that there is no such thing as a "more significant" or "more highly significant" effect. Significance is binary, not scalar: either an effect is declared significant, or it isn't. The whole idea of significance testing is based on this kind of binary decision-making. So $α$ is called the significance level because it's the maximum $p$ allowed for significance. Confidence intervals and their associated confidence levels have completely different motivations, formulations, and interpretations, and not every good confidence interval corresponds to a good significance test, nor vice versa, so it shouldn't be too surprising that the conventions of the two approaches aren't wholly compatible.
• Technical treatments of confidence intervals tend to write the confidence level as $1 - α$ to underscore the analogy to significance testing: the smaller the $α$, the more conservative the analysis.