0
$\begingroup$

I am trying to understand the concept of p-value.

Wikipedia mentions:

The p-value is defined as the probability, under the assumption of hypothesis $H$, of obtaining a result equal to or more extreme than what was actually observed.

Though, I found quite vogue the term extreme, I'm taking it as the values beyond the observed value of the test statistic; by that I could somewhat conceived the definition.

Then, like my book, the wiki article writes:

The smaller the p-value, the larger the significance because it tells the investigator that the hypothesis under consideration may not adequately explain the observation. The hypothesis $H$ is rejected if any of these probabilities is less than or equal to a small, fixed but arbitrarily pre-defined threshold value $\alpha,$ which is referred to as the level of significance [...]

Hmmm... I couldn't get that point; why does the smaller p-value means a higher significance viz, the null hypothesis is on the verge of extinction?

Also, as this site summarises:

  • High P values: your data are likely with a true null.

  • Low P values: your data are unlikely with a true null.

I'm not getting why it is so- why does the lower p-values means the the significance is high?

Wikipedia seems to reason saying this: 'the hypothesis under consideration may not adequately explain the observation'. Can anyone tell me why it is so?

Please help me explaining why lower the p-value, higher is the significance.

Improve this question
$\endgroup$
2

2 Answers 2

Reset to default
2
$\begingroup$

Let's say we have a null hypothesis that the mean of some variable is $0.$ We can draw a random sample from our population and in that sample we find a mean of $1.2\,.$ This could be because our hypothesis is wrong or because we drew a random sample, and the sample will deviate a bit from the population. With (two-sided) statistical testing we compute the probability of drawing a sample with a mean more than or equal to $1.2$ or less than or equal to $-1.2\, .$ If this probability is small then it is unlikely (but not impossible) that the mean of $1.2$ we found in our sample is due to sampling. Since this randomness due to sampling was our only way to "save" the null-hypothesis, we use that small $p$-value as evidence against the null-hypothesis.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for answering; you said If this probability is small then it is unlikely (but not impossible) that the mean of 1.2 we found in our sample is due sampling- why is it so? Why does low p-value imply it was not due to sampling fluctuation? $\endgroup$
    – user74724
    Commented Mar 23, 2016 at 10:06
  • 1
    $\begingroup$ It still possible that the null hypothesis is true and the value 1.2 was due to sampling fluctuation, the probability of drawing such a "weird" sample when the null hypothesis is true is just small. At some point we just say that that probability is too small, and conclude that we reject the null hypothesis. Where that point lies is arbitrary (5% is custormary in many fields), and regardless of that arbitrary point we could still draw the wrong conclusion. $\endgroup$
    – Maarten Buis
    Commented Mar 23, 2016 at 12:36
  • $\begingroup$ Please add this in the answer. $\endgroup$
    – user74724
    Commented Mar 23, 2016 at 12:39
  • $\begingroup$ This is my thinking: if the deviation from hypothesised parameter is due to fluctuation, it would appear more as it is random. But when it is not due to randomness, it would occur otherwise. Am I right, sir? $\endgroup$
    – user74724
    Commented Mar 23, 2016 at 12:41
  • 2
    $\begingroup$ The sample is random. All I am saying that it is hard to distinguish between a random and structural deviation from the null hypothesis if you have a single sample. We can say that large differences are less likely to be random, and the p value quantifies what "large" is. $\endgroup$
    – Maarten Buis
    Commented Mar 23, 2016 at 22:51
0
$\begingroup$

The P-value is the probability of having observed a value as many standard errors away from your hypothesised true value, under the null, given that the null is true. So if I have a P value of 0.05 for my test statistic for the null that the true coefficient is zero, I can say that, if the true coefficient was indeed zero, there is an approximately a 5% probability of my having observed a result that extreme or more extreme.

Improve this answer
$\endgroup$