I know what the significance level is and how hypothesis testing works, so this is primarily a terminological question.
In hypothesis testing, the so-called "significance level" $\alpha$ equals the probability of falsely rejecting a true null hypothesis. If this value is particularly low for a given test for which the null hypothesis has been rejected, this means that the result is very unlikely to have been obtained by chance alone. This yields the very counterintuitive result that when considering a given hypothesis test and the associated decision of rejection or non-rejection of the null hypothesis, a lower significance level means a more highly significant result. The lower the significance level, the higher the significance of the result...
Intuitively, one would expect "significance level" to mean something similar to "confidence level." Unfortunately, it means the opposite. I find that this confuses undergraduates quite a bit and it would be helpful to have a good explanation for them as to how naming $\alpha$ the significance level makes sense.
So, what are the historical, statistical, or mathematical reasons that the significance level inversely related to the confidence level?