Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Based on the correlation of two measures in the following plot:
The p-value tells there is a significant correlation between the two measures but the correlation coefficient R is close to zero that means there is no evidence of any relationship.
I'm confused whether I accept the null hypothesis (there is no relation) or reject it. Based on the p-value, I should reject it but I think it makes sense to accept it since the R value tells us there is no relation.
Can you please explain that to me?
Here’s a key point about the p-value.
It does not quantify by how much your null hypothesis is wrong.
You could have a very subtle effect that is detected by having many observations.
That’s what happened to you. Your data have some slight correlation, but it’s extremely unlikely that it’s due to chance. You’ve detected a real feature of your population, just a subtle one that might not interest you.
there is no relation between measure x and measure y
$\endgroup$
Aug 20, 2019 at 13:46
there is no correlation between measure x and measure y
$\endgroup$
Aug 20, 2019 at 14:51
A technical note: you never accept a null hypothesis based on a test. You either reject it, or you fail to reject it.
A p-value does not tell you which of two hypotheses (null or alternate) is correct. It tells you the probability of finding a more extreme value assuming that no effect exists (the null hypothesis), conditional on some large and important assumptions. By convention, we say that if this probability is less than 5%, we reject the null hypothesis. It does not mean that the null hypothesis is true if the p-value is more than 5%. For that matter, it also does not mean that the alternate hypothesis is true if the p-value is less than 5% - recall that the 5% cutoff is a convention.
A p-value of 0.051 is almost the same as 0.049, so it would be rather silly to assume that the first value means that the null hypothesis is true while the second means that the null hypothesis is false.
The other commentors and @Dave's answer is correct in noting that with a sufficiently large sample size, you are very likely to find a low and significant p-value. This does not tell you much that is useful, but it should tell you that you should be thinking about more than p-values and statistical significance. What is the goal of your analysis?
I'd recommend reading some introductory texts for a more detailed explanation of what these concepts mean - they are important to understand for any analysis you are likely to do or read about.
Did you try plotting the data in log-log scale? It may be that the small correlation becomes very discernible on a log-log scale. Just remember to add 1 when log-transforming: x -> log(x+1)
I don’t think your question here is about whether a small correlation matters - I think you want to know how come your software finds a correlation that you don’t think it’s really visible. Try the above (and it would be appreciated if you could add the corresponding plot here)
Some good answers have been already posted, so I will just post a remark here that was important to me some time ago, that aims to be very intuitive and pragmatical.
When you have a situation like this, think about the opposite case. You have a sample and you estimate a mean (for example) which turns out to be terribly big. So, at first glance, you would say that this estimate reveals that the population mean is actually different from 0, because the sample mean is terribly $big$. However, what does $big$ mean in statistical terms? Or, analogously, what about the standard error? Is it $big$ when adjusted for the standard error?
The test will incorporate the effect of the standard error in the judgement of the estimated mean. So if the standard error is too high, then the test will tell you that we cannot reject the null, even if the estimated mean seems to be large. Because the test will relativize the value of the estimated mean and will compare it to the standard error. So if you have too much standard error then the high standard error means that the estimated mean is too noisy to draw a statistical robust conclusion that the population has a mean which is different from 0.
Here, as long as I have actually understood your point, you have the opposite case: the estimated mean is very low, but the fact that you have a very low standard error reflecting a very low population standard deviation around the true mean, allows you to draw a statistically-motivated inference on the population mean being non-zero (with a certain probability of error in this inference depending on the significance level). In other words, the test will always interprets the estimated value of the mean in light of the corresponding standard error to formulate a decision.
So a $big/small$ estimated mean may mean nothing, if not compared to the standard error
