Why is the p-value of Pearson's correlation test large even when the sample size is sufficient?

Question

I have two variables $Y1$ and $Y2$ and the sample size is $240$. I computed Pearson's correlation coefficient in python with pearsonr from scipy.stats. The following are my results:

Correlation coefficient = 0.0434, p-value=0.4687

The correlation coefficient indicates that there no correlation between $Y1$ and $Y2$, but I am unable to correctly interpret what the p-value means here. Is the p-value large because the correlation coefficient is low? From my understanding, since the p-value>0.05, we cannot reject the null hypothesis ("There is no correlation between the variables"). In this case, are both the p-value and correlation coefficient saying the same thing i.e there is no correlation?

For sufficiently large data sets, will high pearson correlation coefficients also generally have low p-values?

Answer 1

The p-value is a function of both the effect size and the sample size. If you have a gigantic sample size but a tiny (or zero) effect size, then you still do not have to wind up with small p-values.

In the case where the null hypothesis really is true and there is no correlation, the p-value should be $\le0.05$ only $5\%$ of the time. If the null is only slightly incorrect, then even a small sample size might result in $6\%$ or $12\%$ of the p-values being $\le0.05$.

Answer 2

"Is the p-value large because the correlation coefficient is low?"

Basically, yes. A lower sample statistic (r in your example) will always give you a higher p-value, all else being equal.

What this p value is telling you is that the probability that you would get a sample correlation of 0.0434 (or higher) if the actual population correlation is zero (i.e., the null hypothesis is correct) is 0.4687. Because this is a relatively high probability - and is certainly greater than the standard cutoff of 0.05 - you don't have evidence that allows you to reject the null hypothesis.

Try this (I hope it doesn't confuse you more!):

Suppose instead of a probability, you had a certainty: If the population correlation was zero, then the probability of getting a sample correlation of x or lower is 1.0. So, you got a sample correlation of x. I hope you can clearly see that this doesn't give us evidence to reject the antecedent condition (population corr = 0), because we are certain to get this sample correlation when our population correlation is zero. (It also doesn't let us ACCEPT the antecedent. There might be some other underlying circumstances that could also cause a sample correlation of x.)

Now, think about a certainty the other way: If the population correlation is zero, the probability of getting a sample correlation of y is 0. If you get a sample correlation of y, then if this were true you would know that the antecedent cannot be true: Because it is impossible to get that sample correlation if the population correlation is really y. So, you can reject the null hypothesis: We would know that the population correlation must not be zero.

Now, in real life, we don't have certainty. The probability we get is going to be somewhere between 0 & 1. When the p-value we get is below some arbitrarily chosen low cutoff (often 0.05), we essentially treat it as if the probability were zero. In that case, we can reject the null hypothesis.

On the other hand, if the p-value is above that cutoff, then we can't treat it as zero, so we cannot reject our null hypothesis.

Your probability value is not below the cutoff, so you cannot reject the hypothesis that the population correlation is zero.