1
$\begingroup$

I am not sure if it even has a name, but when we look at the p-value for example from scipy.stats.pearsonr what is the test name that is used to test for the null hypothesis?

$\endgroup$
2
$\begingroup$

There's more than one possible test of a Pearson correlation coefficient.

Generically I'd just call it a test of a Pearson correlation.

The test in scipy.stats.pearsonr is predicated on the assumption of joint normality and is for testing the specific null that the population correlation is 0 against either a one- or two-tailed alternative (typically the second).

This would result in a test based on a symmetric beta distribution for the sample correlation under $H_0$ but it can be turned into a test based on a t-distribution (and usually is, but by the look of it scipy.stats.pearsonr sticks with the direct calculation based on the symmetric beta).

You might call it a test of a null Pearson correlation under joint normality, perhaps.

As noted in the Wikipedia article at the above link, nonparametric tests of a null of 0 correlation are possible (e.g. it described a permutation test). Under some sufficiently defined circumstances, other parametric tests (besides the one for joint normality) are possible as well.

There's also tests for the case where the correlation specified under the null is not zero (again, see the Wikipedia link); these can be based on the exact distribution or one based one Fisher's z-transform, but neither of these appear to be implemented in scipy.stats.pearsonr.

$\endgroup$
  • $\begingroup$ It would be worth knowing what the problem with the answer is that led to the downvote. $\endgroup$ – Glen_b -Reinstate Monica Nov 29 '19 at 1:18
  • $\begingroup$ This was informative. In context of time series (where each of the sample is not iid) I read that bartlett's theorem is employed for stationary series to get the corrected confidence intervals. Though that result is asymptotic. But can we say that Bartlett's theorem is also a test of correlation for a more generalized case? Also, could you please cite some formal source for the tests you have talked about in your answer? $\endgroup$ – Dayne Nov 29 '19 at 6:19
  • $\begingroup$ If you check the linked Wikipedia page in the section on "Inference" there are references for the various tests. When you see a number like so$^{[16]}$ in the text , if you hover over it, a reference will pop up. $\endgroup$ – Glen_b -Reinstate Monica Nov 29 '19 at 8:49
2
$\begingroup$

To find the statistical significance of a Pearson r correlation use Fisher's z transformation and see if the confidence intervals includes your hypothesized value. For example, the two-tailed 95% confidence interval of a Pearson r is

$$0.5\ln\mid{\frac{1-r}{1+r}}\mid\pm1.96\frac{1}{\sqrt{n-3}}$$

$\endgroup$
  • $\begingroup$ Perhaps you should write out Fisher's z transformation, as that makes the answer self-contained rather than requiring the reader to look up something in order to understand it. $\endgroup$ – jbowman Nov 28 '19 at 22:27
  • $\begingroup$ @jbowman edited $\endgroup$ – Jay Schyler Raadt Nov 29 '19 at 1:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.