Suppose that I have two datasets
import pandas as pd
import numpy as np
import scipy.stats as st
x = [10, 15, 20, 50, 100]
y = [20, 31, 38, 51, 97]
I know that it is possible to compute the correlation $\rho$ between these sets and a 95% confidence interval with a Fisher transform, as follows:
r, p = st.pearsonr(x,y)
r_z = np.arctanh(r)
se = 1/np.sqrt(x.size-3)
alpha = 0.05
z = st.norm.ppf(1-alpha/2)
lo_z, hi_z = r_z-z*se, r_z+z*se
lo, hi = np.tanh((lo_z, hi_z))
However I have two questions:
I believe, in the above methodology, the underlying null hypothesis is $\rho = 0$ and the alternative hypothesis is $\rho \neq 0$. However, in my case, the prior is that the two series $x,y$ are correlated with $\rho = 1$ and my alternative is that $\rho \neq 1$
Second point, the size of my data are usually quite small, say between 4 and 15 data points. Given that I know the Fisher transform is an asymptotic approximation for large $n$, is there a better method than above for calculating the confidence interval I want?