# Significance Levels, Confidence Intervals and P-Values

I'm new to statistics and still learning some of the basic concepts. Although I am slowly getting them sorted out, I still often find myself getting confused between Significance Levels, Confidence Intervals and P-Values. Just wondering if anyone can help with a refreshing, clear explanation of how they are each used and perhaps related?

• Take a look at this web page: onlinestatbook.com/2/logic_of_hypothesis_testing/sign_conf.html Mar 31, 2017 at 22:11
• A p-value is the probability of obtaining a value as extreme or more extreme than the observed value when the null hypothesis is assumed. Confidence intervals are intervals that in repeated sampling would include the true parameter a certain percentage of time. The significance level is the percentage of the time that the confidence interval will not contain the parameter. This all depends on the null hypothesis and the assumed family of probability distributions. David Lane's online text provides more detail. Mar 31, 2017 at 22:27
• Thanks for the feedback guys, this was very helpful. David Lane's page looks like a great resource too, looking forward to digging into it some more. Apr 1, 2017 at 5:55
• Check out statquest videos on YouTube for a clear explanation of these concepts without getting into confusing details Jul 24, 2020 at 15:57
• Any attempt to explain the nature of and relationships between those things should be set within a discussion of the nature of statistical and scientific inferences. I grew frustrated with the context-free discussions in the P-value controversies and so I wrote an extensive chapter on it. Read it and you will not be disappointed: link.springer.com/chapter/10.1007/164_2019_286 Sep 26, 2021 at 21:10

Confidence intervals (only) provide a plausible range for population parameters. For instance, the population proportion would be within the range $$(p_1, p_2)$$ with 95% confidence level. That's if we take $$N$$ samples of the same sample size, 95% of the time the sample proportion would be within that range. Note that 95% is the confidence level. And the significance level, often denoted by $$\alpha$$, is the complement of the confidence level only when the hypothesis testing is two sided. That is $$\text{confidence level} = 1 - \text{significance level}$$ in that scenario. For the one sided hypothesis testing $$\text{confidence level} = 1 - 2*\text{significance level}$$. The area of the confidence interval is always symmetry.
The $$\text{significance level} = P(\text{Type 1 error} | H_0 \text{true})$$, and the type 1 error implies we reject the null hypothesis wrongly, then the bigger the significance level the more likely we would make a false rejection.
We suppose that the null hypothesis is true and the p-value represents the probability the extreme(or more extreme) sample proportion is observed by chance: $$P(\text{observed or more extreme sample statistic}|H_0 \text{true})$$. The bigger the p-value the more likely the observed sample proportion is resulted by chance and the less we would reject the null hypothesis, but the rejection depends on the significance level we set beforehand. If the p-value is smaller than the significance level we would be confident enough that the observed sample proportion is not led by chance and we would reject the null hypothesis.