0
$\begingroup$

In this image we see that the deviations of the null hypothesis are at the edges of the distribution. The blue area denotes the probability to track this deviation. In my opinion, if the area gets larger than a threshold, then the probability to have those deviations increases, and thus we must reject the null hypothesis.

But I read that we reject the null hypothesis if p-value is lower than this threshold. If p-value corresponds to this area then something I misunderstand.

Can you help me clarify this?

enter image description here

Improve this question
$\endgroup$
7
  • $\begingroup$ I feel it deserves more than a simple answer. I encourage you to keep going with your work. The image is a bit misleading: "Observed p-value (significance level)" because they are not the same. The significance level is the probability of the study to reject the null hypothesis. It is a threshold that you fixed at the beginning of your study (often 0.05) while the p-value is the probability of obtaining a result at least as extreme, given that the null hypothesis is true (en.wikipedia.org/wiki/Statistical_significance). They are both probabilities. $\endgroup$
    – Pitouille
    Sep 29, 2021 at 7:33
  • $\begingroup$ "But I read that we reject the null hypothesis if p-value is lower than this threshold." --- read where, discussing what circumstance? The critical region (and so the regions where you look to get a p-value) may be in the upper tail, the lower tail, both tails -- even the middle of the distribution, in appropriate circumstances, or even in several different subsets of the support of the variable. It depends on what's most consistent with the alternative (/relatively most consistent, compared to the null). $\endgroup$
    – Glen_b
    Sep 29, 2021 at 7:54
  • $\begingroup$ Glen_b and Pitouille thank you for ansers.I still dont get though,(at least it is not clear to me) if the blue area in the diagramm corresponds to the p valueAnd if it corresponds why if growing(deviations grow too) ,we accept the null hypothesis...That was my main issue $\endgroup$
    – THEODOROPOULOS DIMITRIS
    Sep 29, 2021 at 8:56
  • $\begingroup$ You never accept the null hypothesis, you only potentially reject it, if the p-value is small enough. If you don't reject the null then you conclude that you could not reject it (given your data and model). $\endgroup$
    – user2974951
    Sep 29, 2021 at 8:59
  • $\begingroup$ I am not sure what do you mean about "deviations grow too"... You might want to have a look at: stats.stackexchange.com/a/130772/328636. I encourage you also to look at other resources since this image might be confusing. The green dot (observed result) is actually the result of your statistic test... which leads to a p-value which corresponds to a probability (hence the region). Nothing can really grow here, you significance level is fixed (I mean you decide what it should be early in the study) and the statistic test result depends on a formula. So, what do you mean by "if growing"? $\endgroup$
    – Pitouille
    Sep 29, 2021 at 12:04

2 Answers 2

Reset to default
4
$\begingroup$

If I got it right, I think you are struggling with rejecting or not the null hypothesis. I will take a simple example of people's height in cm for a fictious country for which we know the population standard deviation. Therefore, we can use the z-test to illustrate the image you shared. I am going to focus mainly on your question and assume that you understand how the test statistics is "built".

Let's assume that we have a random sample of size $n = 50$ from this fictious country. From this sample, you want to make inferences on the broader population it belongs to. For example, you want to determine whether the population mean is different from $\mu_0 = 170$ (for the sake of the exercise).

In the z-test $\frac{\overline{X} - \mu_0}{\sigma/\sqrt{n}}$, $\overline{X}$ is actually a random variable (uppercase notation) which represents the sampling distribution of the sample means. To understand it, the sample you got is only one possible sample out of the many we could have drawn from our population. If we imagine that we could get a significant number of random samples (of the same size) from our population of interest, we would be able to calculate the mean for each of them. This distribution of sample means is the sampling distribution of the means.

The Central Limit Theorem states that, given a sufficiently large sample size, the sampling distribution of the mean for a variable will approximate a normal distribution. We can empirically illustrate that by simulating this sampling distribution of the mean through bootstrapping :

enter image description here

$\frac{\overline{X} - \mu_0}{\sigma/\sqrt{n}}$ follows a standard normal distribution. It is centered on the null hypothesis $\mu_0$. When we inject the mean of our sample $\bar x$ (lowercase notation) in the z-test formula, we will be able to see how far we are from $\mu_0$. For instance, a value greater than 1.96 or lower than -1.96 will indicate that we are more than 2 standard deviations away from $\mu_0$. If you remember also the properties of the standard normal distribution, you know that the probability to get such values represents only 5% of the distribution. In other words, it would mean that to the probability to get a mean as extreme as what we observed from our sample will be unlikely... So we can reject the null hypothesis.

The significance level is the threshold that you fix before doing the test and base on your knowledge of the subject. Of course, if this "threshold" is too "high", then you will be in a situation where you cannot reject the null hypothesis. You can see the significance level as a measure to assess the strength of your analysis.

Improve this answer
$\endgroup$
3
$\begingroup$

A more sophisticated discussion of p-values and hypothesis tests has been given in another answer and I don't dispute its value, however I try to answer the question as straight as I can.

The p-value in the image is the area under the curve to the right of the observed result. The farther on the right the observed result is, the less it is in line with the null hypothesis (or in other words, the farther away from what is expected under the null hypothesis; more generally this can also happen on the left side but in the case illustrated by the image it's on the right). For this reason, observed results less likely under the null hypothesis, i.e., farther away from the center and here more extremely on the right, are a reason to reject the H0. But the more on the right the observed value is, the smaller becomes the area under the curve to its right (the probability to observe something even more extreme), which is the p-value. For this reason, a small p-value, rather than a large one, leads to rejection.

Making reference to the question: "In my opinion, if the area gets larger than a threshold, then the probability to have those deviations increases" - but if the probability for having "deviations" of a certain kind is large, these are no longer "deviations" that really deviate from what typically happens under the null hypothesis (large probability means they can happen quite often), and therefore they cannot be counted against it. We reject the null hypothesis if something happens that has a low probability under it and is therefore not to be expected (in case the H0 is true).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.