I am not sure if it even has a name, but when we look at the p-value for example from scipy.stats.pearsonr what is the test name that is used to test for the null hypothesis?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
There's more than one possible test of a Pearson correlation coefficient.
Generically I'd just call it a test of a Pearson correlation.
The test in scipy.stats.pearsonr is predicated on the assumption of joint normality and is for testing the specific null that the population correlation is 0 against either a one- or two-tailed alternative (typically the second).
This would result in a test based on a symmetric beta distribution for the sample correlation under $H_0$ but it can be turned into a test based on a t-distribution (and usually is, but by the look of it scipy.stats.pearsonr sticks with the direct calculation based on the symmetric beta).
You might call it a test of a null Pearson correlation under joint normality, perhaps.
As noted in the Wikipedia article at the above link, nonparametric tests of a null of 0 correlation are possible (e.g. it described a permutation test). Under some sufficiently defined circumstances, other parametric tests (besides the one for joint normality) are possible as well.
There's also tests for the case where the correlation specified under the null is not zero (again, see the Wikipedia link); these can be based on the exact distribution or one based one Fisher's z-transform, but neither of these appear to be implemented in scipy.stats.pearsonr.
-
$\begingroup$ It would be worth knowing what the problem with the answer is that led to the downvote. $\endgroup$– Glen_bCommented Nov 29, 2019 at 1:18
-
$\begingroup$ This was informative. In context of time series (where each of the sample is not iid) I read that bartlett's theorem is employed for stationary series to get the corrected confidence intervals. Though that result is asymptotic. But can we say that Bartlett's theorem is also a test of correlation for a more generalized case? Also, could you please cite some formal source for the tests you have talked about in your answer? $\endgroup$– DayneCommented Nov 29, 2019 at 6:19
-
$\begingroup$ If you check the linked Wikipedia page in the section on "Inference" there are references for the various tests. When you see a number like so$^{[16]}$ in the text , if you hover over it, a reference will pop up. $\endgroup$– Glen_bCommented Nov 29, 2019 at 8:49
$\begingroup$
$\endgroup$
2
To find the statistical significance of a Pearson r correlation use Fisher's z transformation and see if the confidence intervals includes your hypothesized value. For example, the two-tailed 95% confidence interval of a Pearson r is
$$0.5\ln\mid{\frac{1-r}{1+r}}\mid\pm1.96\frac{1}{\sqrt{n-3}}$$
answered Nov 28, 2019 at 20:37
-
$\begingroup$ Perhaps you should write out Fisher's z transformation, as that makes the answer self-contained rather than requiring the reader to look up something in order to understand it. $\endgroup$– jbowmanCommented Nov 28, 2019 at 22:27
-