4
$\begingroup$

Say the null hypothesis is that two samples emanate from two distributions with either equal medians or equal variances. And p values are the probability that a value or more extreme comes from the original distribution given the null hypothesis holds. Therefore, why do correlations with low p-values mean that they are strong? It seems that holds if the null hypothesis is that they do NOT come from similar distributions.

Can you please put me straight?

Regards,

$\endgroup$
7
$\begingroup$

The p value is the probability of making a certain observation - or more extreme - due to chance, given that the observation is drawn from the original distribution.

Normally (haha) you would expect observations to be pretty close to the distribution's arithmetic mean. If an observation is far from the mean it can be because of two things: a) the observation occurred due to chance - this has to be possible; otherwise the distribution's probability density function would be 0 away from the mean. Or b) the observation is different because it is generated by a different distribution with a mean that differs from the original one. So a small p value means that an observation that far from the original distribution's mean is super unlikely, if it was generated by that distribution. Totally possible but really unlikely (i.e. with probability p).

Now there is this convention in one branch of the statistics community to consider p values under a certain threshold to be evidence that the observation is from a distribution different from the original one. That threshold is called "alpha" and is set prior to an experiment. So if alpha e.g. 0.05 and p is below that, we think that this is evidence that the observation is from a different distribution.

That's the story for p values for means. When it comes to correlations, the strength of the correlation (which is the observation in this case) is given by a different variable - e.g. r. Having a high r means the correlation is strong. But even high values of r can have different values of p - high ones and low ones. With a low p value we think that it is unlikely to get that kind of r due to chance. A high p value can be sign for having a great amount of noise in the data so that the correlation itself is high but the probability that this is due to chance is also high. You can also have very low values of r with high certainty (low values of p). So the strength of correlations is not measured by p.

$\endgroup$
  • $\begingroup$ Nicely explained. I see, so if I had an experiment and a numerical comparison then my correlation can only be really judged by r and not p? So if an article reported that reduction in X was correlated to reduction in Y (p<0.001), the p value doesn't mean squat? And hence a value for r should be reported instead? Or am I still misunderstanding the concept? $\endgroup$ – HCAI Feb 10 '12 at 13:25
  • 5
    $\begingroup$ No. The $p$ value is not the probability that a given observation or more extreme is from the original distribution. The $p$ value is the probability of a given observation or more extreme from the original distribution. $\endgroup$ – Henry Feb 10 '12 at 15:04
  • 5
    $\begingroup$ The $p$ value is a measure of how extreme the observation is, given the assumption that it comes from a particular distribution. It is not a measure of the probabilities that the observation comes from that distribution or a different distribution. Work out the $p$ value for seeing 115 heads from 200 coin flips of an assumed fair coin (about 0.04) and then note from Wikipedia that since the probability of this exact observation (about 0.006) is higher than 1/201, it does not provide additional evidence of the coin being biased. $\endgroup$ – Henry Feb 10 '12 at 15:38
  • 1
    $\begingroup$ I'm curious (and I truly don't know the answer): if we all examined our lives in research/statistics before and after we "saw the light" (one hopes) about p-values, would many of us find that the latter period was notably more productive as a result? $\endgroup$ – rolando2 Feb 11 '12 at 1:31
  • 2
    $\begingroup$ @user1134241 The p value is not interpreted in a way like "low is good, great is bad". A low p means that it is unlikely to get an observation as the one you made or even more extreme (i.e. deviant from a given value) purely by chance. Period. If you compare your prediction and an observation, a low p value means that "it's unlikely to get this difference purely by chance - so maybe there is a systematic effect going on". A high p means, yeah, this (small) difference is likely due to chance, so don't bother and treat the two observations as resulting from the same distribution. $\endgroup$ – xmjx Mar 14 '12 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.