# Significance of average correlation coefficient

Disclaimer: if you find this question to be too similar to another one, I happy for it to be merged. However, I did not find a satisfactory answer anywhere else (and do not yet have the "reputation" to comment or upvote), so I thought it would be best to ask a new question myself.

My question is this. For each of 12 human subjects, I have computed a correlation coefficient (Spearman's rho) between 6 levels of an independent variable X, and corresponding observations of a dependent variable Y. (Note: the levels of X are not equal across subjects.) My null hypothesis is that in the general population, this correlation is equal to zero. I have tested this hypothesis in two ways:

1. Using a one-sample t-test on the correlation coefficients obtained from my 12 subjects.

2. By centering my levels of X and observations of Y such that for each participant, mean(X) = 0 and mean(Y) = 0, and then computing a correlation over the aggregate data (72 levels of X and 72 observations of Y).

Now, from reading about working with correlation coefficients (here and elsewhere) I have started to doubt whether the first approach is valid. Particularly, I have seen the following equation pop up in several places, presented (apparently) as a t-test for average corelation coefficients:

$$t = \frac{r}{SE_{r}} = \frac{\sqrt{n-2}}{\sqrt{1-r^{2}}}$$

where $r$ would be the average correlation coefficient (and let's assume we've obtained this using Fisher's transformation on the per-subject coefficients first) and $n$ the number of observations. Intuitively, this seems wrong to me as it does not include any measure of the between-subject variability. In other words, if I had 3 correlation coefficients, I would get the same t-statistic whether they were [0.1, 0.5, 0.9] or [0.45 0.5 0.55] or any range of values with the same mean (and $n=3$)

I suspect, therefore, that the above equation does not in fact apply when testing the significance of an average of correlation coefficients, but when testing the significance of a single correlation coefficient based on $n$ observations of 2 variables.

Could anyone here please confirm this intuition or explain why it is wrong? Also, if this formula doesn't apply to my case, does anyone know a/the correct approach? Or perhaps my own test number 2 is already valid? Any help is greatly appreciated (including pointers to previous answers that I may have missed or misinterpreted).

• Pearson's $r$ is insensitive to centering and scaling transformations, so I think centering is irrelevant to your question. For example, cor($X,Y$) = cor($X,Y-\bar{Y}$) = cor($X,Y+1000$) = cor($X,Y\times 1000$). – Alexis Dec 26 '17 at 19:45
• I agree with you. That's why I interpreted centering as "centering each variable separately before putting them together". – Federico Tedeschi Dec 27 '17 at 10:27
• @FedericoTedeschi Isn't "centering each variable separately before putting them together" what $Y-\bar{Y}$ means? – Alexis Dec 29 '17 at 16:13
• @Alexis I have replied to you at the bottom of my answer (it would have been too long to write it in a comment, and I would also have had to correct it several times due to the WYSINWYG problem). – Federico Tedeschi Dec 29 '17 at 20:00

I assume that the $12$ variables ($6$ $X$'s and $6$ $Y$'s) are the same for all individuals (actually I'm not sure I understand what you mean by saying that the levels are not equal across subjects: I hope you are referring about the independence among the ranges of the variables, not about which variables are measured for each individual). Yes, the formula you showed applies to the correlation coefficient between two variables.

In your point 2, you talk about normalizing: I think this would make sense if you did it for each of the $6*2$ variables separately. However, even so, the problem with this approach is that it does not control for within-individual dependency.

I believe your approach 1 is not valid either, because it would be a test among $6$ variables with distribution $t$ with just $10$ degrees of freedom, so I don't think you can apply the Central Limit Theorem in this case.

Maybe, with larger numbers, you could use a random effect approach, allowing for a random slope and simultaneously testing for both a null average coefficient (of $X_i$ on $Y_i$) and non-existence of a random coefficient. I believe however 6 variables and 12 observations are not enough to do it.

I suggest you see it as a test on 6 values (becoming 12 if you also consider values below the diagonal) of the correlation matrix among the $12$ variables (both the $X$ and the $Y$), i.e. those on the diagonal of the 2nd (and equivalently of the 3rd) quadrant. Thus, I would make a likelihood ratio test between the restricted and the unrestricted model.

@Alexis My understanding is that centering $X_1, \dots, X_6$, $Y_1, \dots, Y_6$, by replacing them with $X_1^*=X_1-\bar{X_1}, \dots, X_6^*=X_6-\bar{X_6}, Y_1^*=Y_1-\bar{Y_1}, \dots, Y_6^*=Y_6-\bar{Y_6}$ would make sense (I think it would also make sense to divide them by their $SE$'s). In this way, the variables $X^*$ and $Y^*$ (created by considering the $X_i^*, 1 \leq i \leq 6$ as if they were occurrences of a unique variable, and the same for $Y_i^*$) would all have a $0$ mean. On the contrary, if we build two variables $X, Y$ first (created by considering the $X_i, 1 \leq i \leq 6$ as if they were occurrences of a unique variable, and the same for $Y_i$), then of course subtracting the mean (and also dividing by the SE of $X$ and $Y$) would not change things.

EDIT 01/01/18

Let $i$ indicate the variable and $j$ ($1\leq j\leq 12$) the individual. Then, suppose we have:

$X_{1j}=Y_{1j}=10, \forall j$;

$X_{2j}=Y_{2j}=8, \forall j$;

$X_{3j}=Y_{3j}=6, \forall j$;

$X_{4j}=Y_{4j}=4, \forall j$;

$X_{5j}=Y_{5j}=2, \forall j$;

$X_{6j}=-Y_{6j}=j, \forall j$.

The correlation in this case should be $0.5428$.

If we center each variable, given that, for $1 \leq i \leq 5$, both $X_i$ and $Y_i$ have no variation, we have: $X_{ij}^*=Y_{ij}^*=0$. As for $i=6$, we get the values $X_{6j}^*=j-6.5, Y_{j6}^*=(13-j)-6.5=6.5-j$ (i.e., for the $X$'s: $-5.5, -4.5, -3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5, 4.5, 5.5$, and exactly the opposite for the $Y$'s). Since $0=-0$ and $j-6.5=-(6.5-j)$, we get: $X_{ij}^*=-Y_{ij}^* \forall i,j \rightarrow X^*=-Y^*$, implying a correlation of $-1$.

• I agree with you, if we follow the second procedure. That's why I believe Ruben van Bergen meant what I described in the 1st procedure. In this case, we have that: $cor(X_i,Y_i)=cor(X_i^*,Y_i^*), \forall i$, but $cor(X,Y)=cor(X^*,Y^*)$ is not generally true. I am editing my post to show a counter-example. – Federico Tedeschi Jan 1 '18 at 17:15
• The values giving a correlation of $0.5428$ are: $X=10,10,10,10,10,10,10,10,10,10,10,10,8,8,8,8,8,8,8,8,8,8,8,8,6,6,6,6,6,6,6,6,6,6,6,6,4,4,4,4,4,4,4,4,4,4,4,4,2,2,2,2,2,2,2,2,2,2,2,2,1,2,3,4,5,6,7,8,9,10,11,12$; $Y=10,10,10,10,10,10,10,10,10,10,10,8,8,8,8,8,8,8,8,8,8,8,8,6,6,6,6,6,6,6,6,6,6,6,6,4,4,4,4,4,4,4,4,4,4,4,4,2,2,2,2,2,2,2,2,2,2,2,2,12,11,10,9,8,7,6,5,4,3,2,1$. It doesn't matter whether the correlation is really $0.5428$, since it is clearly different from $-1$. – Federico Tedeschi Jan 2 '18 at 13:44
• The correlation between $X^*=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-5.5,-4.5,-3.5,-2.5,-1.5,-0.5,0.5,1.5,2.5,3.5,4.5,5.5$ and $X^*=0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5.5,4.5,3.5,2.5,1.5,0.5,-0.5,-1.5,-2.5,-3.5,-4.5,-5.5$ is $-1$. The fact you say that $X=1,\dots, 12$ and $Y=12, \dots, 1$ leads to $cor(X,Y)=cor(X^*,Y^*)=-1$ is true, but this only means that $cor(X_i,Y_i)=cor(X^*_i,Y^*_i)$, that is something I've already written. – Federico Tedeschi Jan 2 '18 at 15:41
• Of course $cor(X;Y)=cor(X-\bar{X};Y-\bar{Y})$: this is a consequence of the invariance of correlation to linear transformations. This is something I have agreed upon already in my first comment, "I agree with you. That's why I interpreted centering as "centering each variable separately before putting them together". "– Federico Tedeschi Dec 27 '17 at 10:27 – Federico Tedeschi Jan 2 '18 at 16:54
• Perhaps, I do not understand what "centering each variable separately before putting them together" means. To me, $X - \bar{X}$ means $X_{1} - \bar{X}, X_{2}-\bar{X},\dots, X_{n}-\bar{X}$ is "centering each variable separately before putting them together". Can you help me understand our apparent different understanding? – Alexis Jan 2 '18 at 17:05