10
$\begingroup$

I want to test a sample correlation $r$ for significance ($n=16$), using p-values, in Python. As I have understood from this question, I can achieve that by using Fisher's z-transform.

Is there a Python module, which allows easy use of Fisher's z-transform?

I have not been able to find the functionality in SciPy or Statsmodels. So far, I have had to write my own messy temporary function:

import numpy as np
from scipy.stats import zprob
def z_transform(r, n):
    z = np.log((1 + r) / (1 - r)) * (np.sqrt(n - 3) / 2)
    p = zprob(-z)
    return p
Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ AFAIK the Fisher transform equals the inverse hyperbolic tangent, so just use that. $\endgroup$
    – jona
    Commented Jul 23, 2014 at 14:23
  • $\begingroup$ Presumably z-transform is a typo, since that's an invalid Python identifier (- is interpreted as subtraction). $\endgroup$
    – ali_m
    Commented Jul 7, 2015 at 11:08
  • $\begingroup$ This test assumes that you're sampling from a bivariate normal distribution. Does that make sense here? I'd prefer to do some sort of randomization test and approximate the null distribution using simulation. $\endgroup$
    – dsaxton
    Commented Jul 7, 2015 at 21:08

1 Answer 1

Reset to default
5
$\begingroup$

The Fisher transform equals the inverse hyperbolic tangent‌​/arctanh, which is implemented for example in numpy. The inverse Fisher transform/tanh can be dealt with similarly.

Moreover, numpy's function for Pearson's correlation also gives a p value.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ The pearsonr() function in numpy is "only reasonable for datasets larger than 500". $\endgroup$
    – dwitvliet
    Commented Jul 23, 2014 at 15:41
  • 1
    $\begingroup$ Reading the sources, numpy uses the t score to test the significance of the correlation. I'm not aware of this approach being especially problematic. $\endgroup$
    – jona
    Commented Jul 23, 2014 at 15:47
  • 1
    $\begingroup$ Check if the p-values from pearsonr() and the z-transformed approach are similar! $\endgroup$
    – jona
    Commented Jul 23, 2014 at 17:32
  • $\begingroup$ Both pearsonr and np.arctanh take only a single row or column array as input, not arrays with more than one row or column. $\endgroup$
    – m13op22
    Commented Apr 13, 2019 at 5:10
  • 1
    $\begingroup$ To test the significance of the difference between two correlation coefficients, r1 and r2, how can i do that? I need to first convert r-to-z and then take the difference to see the z-score effect size? $\endgroup$
    – seralouk
    Commented Aug 8, 2022 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.