2
$\begingroup$

Based on the correlation of two measures in the following plot:

enter image description here

The p-value tells there is a significant correlation between the two measures but the correlation coefficient R is close to zero that means there is no evidence of any relationship.

I'm confused whether I accept the null hypothesis (there is no relation) or reject it. Based on the p-value, I should reject it but I think it makes sense to accept it since the R value tells us there is no relation.

Can you please explain that to me?

$\endgroup$
4
  • $\begingroup$ How big is your sample? $\endgroup$ Commented Aug 20, 2019 at 11:23
  • $\begingroup$ @user2974951 9930 points $\endgroup$
    – Adam Amin
    Commented Aug 20, 2019 at 11:23
  • 1
    $\begingroup$ Your sample is big enough that you will reject the null most times. This is now a question of practical significance. Is -0.16 of practical importance to you? If not, you can treat it as being zero. $\endgroup$ Commented Aug 20, 2019 at 11:25
  • $\begingroup$ I think no meaningful answer can be given until you specify what your two quantities (the values on the $x$ and $y$ axes) are about. $\endgroup$
    – pglpm
    Commented Aug 20, 2019 at 13:30

4 Answers 4

2
$\begingroup$

Here’s a key point about the p-value.

It does not quantify by how much your null hypothesis is wrong.

You could have a very subtle effect that is detected by having many observations.

That’s what happened to you. Your data have some slight correlation, but it’s extremely unlikely that it’s due to chance. You’ve detected a real feature of your population, just a subtle one that might not interest you.

$\endgroup$
7
  • $\begingroup$ Thanks. In this case, the null hypothesis is rejected, right? $\endgroup$
    – Adam Amin
    Commented Aug 20, 2019 at 11:47
  • $\begingroup$ @AdamAmin It depends on your null hypothesis. In your case, the software most likely is testing $\rho=0$, so there would be an emphatic rejection. $\endgroup$
    – Dave
    Commented Aug 20, 2019 at 12:35
  • $\begingroup$ My null hypothesis is there is no relation between measure x and measure y $\endgroup$
    – Adam Amin
    Commented Aug 20, 2019 at 13:46
  • $\begingroup$ Then you're asking the wrong question. There are a lot of ways that data can be related ("dependent" in statistics lingo, though causation is not implied by this term). Correlation is only one of those ways. If your scatterplot formed a ring, you'd conclude that there is no correlation but that there most certainly is a relationship! $\endgroup$
    – Dave
    Commented Aug 20, 2019 at 13:50
  • $\begingroup$ I'm sorry, I'm still confused. Does that mean in my case there is a correlation but there is no relationship between the two measures? What happens if the null hypothesis says there is no correlation between measure x and measure y $\endgroup$
    – Adam Amin
    Commented Aug 20, 2019 at 14:51
0
$\begingroup$

A technical note: you never accept a null hypothesis based on a test. You either reject it, or you fail to reject it.

A p-value does not tell you which of two hypotheses (null or alternate) is correct. It tells you the probability of finding a more extreme value assuming that no effect exists (the null hypothesis), conditional on some large and important assumptions. By convention, we say that if this probability is less than 5%, we reject the null hypothesis. It does not mean that the null hypothesis is true if the p-value is more than 5%. For that matter, it also does not mean that the alternate hypothesis is true if the p-value is less than 5% - recall that the 5% cutoff is a convention.

A p-value of 0.051 is almost the same as 0.049, so it would be rather silly to assume that the first value means that the null hypothesis is true while the second means that the null hypothesis is false.

The other commentors and @Dave's answer is correct in noting that with a sufficiently large sample size, you are very likely to find a low and significant p-value. This does not tell you much that is useful, but it should tell you that you should be thinking about more than p-values and statistical significance. What is the goal of your analysis?

I'd recommend reading some introductory texts for a more detailed explanation of what these concepts mean - they are important to understand for any analysis you are likely to do or read about.

$\endgroup$
0
$\begingroup$

Did you try plotting the data in log-log scale? It may be that the small correlation becomes very discernible on a log-log scale. Just remember to add 1 when log-transforming: x -> log(x+1)

I don’t think your question here is about whether a small correlation matters - I think you want to know how come your software finds a correlation that you don’t think it’s really visible. Try the above (and it would be appreciated if you could add the corresponding plot here)

$\endgroup$
0
$\begingroup$

Some good answers have been already posted, so I will just post a remark here that was important to me some time ago, that aims to be very intuitive and pragmatical.

When you have a situation like this, think about the opposite case. You have a sample and you estimate a mean (for example) which turns out to be terribly big. So, at first glance, you would say that this estimate reveals that the population mean is actually different from 0, because the sample mean is terribly $big$. However, what does $big$ mean in statistical terms? Or, analogously, what about the standard error? Is it $big$ when adjusted for the standard error?

The test will incorporate the effect of the standard error in the judgement of the estimated mean. So if the standard error is too high, then the test will tell you that we cannot reject the null, even if the estimated mean seems to be large. Because the test will relativize the value of the estimated mean and will compare it to the standard error. So if you have too much standard error then the high standard error means that the estimated mean is too noisy to draw a statistical robust conclusion that the population has a mean which is different from 0.

Here, as long as I have actually understood your point, you have the opposite case: the estimated mean is very low, but the fact that you have a very low standard error reflecting a very low population standard deviation around the true mean, allows you to draw a statistically-motivated inference on the population mean being non-zero (with a certain probability of error in this inference depending on the significance level). In other words, the test will always interprets the estimated value of the mean in light of the corresponding standard error to formulate a decision.

So a $big/small$ estimated mean may mean nothing, if not compared to the standard error

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.