Significance of average correlation coefficient

Question

Disclaimer: if you find this question to be too similar to another one, I happy for it to be merged. However, I did not find a satisfactory answer anywhere else (and do not yet have the "reputation" to comment or upvote), so I thought it would be best to ask a new question myself.

My question is this. For each of 12 human subjects, I have computed a correlation coefficient (Spearman's rho) between 6 levels of an independent variable X, and corresponding observations of a dependent variable Y. (Note: the levels of X are not equal across subjects.) My null hypothesis is that in the general population, this correlation is equal to zero. I have tested this hypothesis in two ways:

1. Using a one-sample t-test on the correlation coefficients obtained from my 12 subjects.

2. By centering my levels of X and observations of Y such that for each participant, mean(X) = 0 and mean(Y) = 0, and then computing a correlation over the aggregate data (72 levels of X and 72 observations of Y).

Now, from reading about working with correlation coefficients (here and elsewhere) I have started to doubt whether the first approach is valid. Particularly, I have seen the following equation pop up in several places, presented (apparently) as a t-test for average corelation coefficients:

$$t = \frac{r}{SE_{r}} = \frac{\sqrt{n-2}}{\sqrt{1-r^{2}}}$$

where $r$ would be the average correlation coefficient (and let's assume we've obtained this using Fisher's transformation on the per-subject coefficients first) and $n$ the number of observations. Intuitively, this seems wrong to me as it does not include any measure of the between-subject variability. In other words, if I had 3 correlation coefficients, I would get the same t-statistic whether they were [0.1, 0.5, 0.9] or [0.45 0.5 0.55] or any range of values with the same mean (and $n=3$)

I suspect, therefore, that the above equation does not in fact apply when testing the significance of an average of correlation coefficients, but when testing the significance of a single correlation coefficient based on $n$ observations of 2 variables.

Could anyone here please confirm this intuition or explain why it is wrong? Also, if this formula doesn't apply to my case, does anyone know a/the correct approach? Or perhaps my own test number 2 is already valid? Any help is greatly appreciated (including pointers to previous answers that I may have missed or misinterpreted).

Answer 1

A better approach to analysing this data is to use a mixed-model (a.k.a. mixed effects model, hierarchical model) with subject as a random effect (random intercept or random intercept + slope). To summarize a different answer of mine:

This is essentially a regression that models a single overall relationship while allowing that relationship to differ between groups (the human subjects). This approach benefits from partial pooling and uses your data more efficiently.

Answer 2

I assume that the $12$ variables ($6$ $X$'s and $6$ $Y$'s) are the same for all individuals (actually I'm not sure I understand what you mean by saying that the levels are not equal across subjects: I hope you are referring about the independence among the ranges of the variables, not about which variables are measured for each individual). Yes, the formula you showed applies to the correlation coefficient between two variables.

In your point 2, you talk about normalizing: I think this would make sense if you did it for each of the $6*2$ variables separately. However, even so, the problem with this approach is that it does not control for within-individual dependency.

I believe your approach 1 is not valid either, because it would be a test among $6$ variables with distribution $t$ with just $10$ degrees of freedom, so I don't think you can apply the Central Limit Theorem in this case.

Maybe, with larger numbers, you could use a random effect approach, allowing for a random slope and simultaneously testing for both a null average coefficient (of $X_i$ on $Y_i$) and non-existence of a random coefficient. I believe however 6 variables and 12 observations are not enough to do it.

I suggest you see it as a test on 6 values (becoming 12 if you also consider values below the diagonal) of the correlation matrix among the $12$ variables (both the $X$ and the $Y$), i.e. those on the diagonal of the 2nd (and equivalently of the 3rd) quadrant. Thus, I would make a likelihood ratio test between the restricted and the unrestricted model.

@Alexis My understanding is that centering $X_1, \dots, X_6$, $Y_1, \dots, Y_6$, by replacing them with $X_1^*=X_1-\bar{X_1}, \dots, X_6^*=X_6-\bar{X_6}, Y_1^*=Y_1-\bar{Y_1}, \dots, Y_6^*=Y_6-\bar{Y_6}$ would make sense (I think it would also make sense to divide them by their $SE$'s). In this way, the variables $X^*$ and $Y^*$ (created by considering the $X_i^*, 1 \leq i \leq 6$ as if they were occurrences of a unique variable, and the same for $Y_i^*$) would all have a $0$ mean. On the contrary, if we build two variables $X, Y$ first (created by considering the $X_i, 1 \leq i \leq 6$ as if they were occurrences of a unique variable, and the same for $Y_i$), then of course subtracting the mean (and also dividing by the SE of $X$ and $Y$) would not change things.

EDIT 01/01/18

Let $i$ indicate the variable and $j$ ($1\leq j\leq 12$) the individual. Then, suppose we have:

$X_{1j}=Y_{1j}=10, \forall j$;

$X_{2j}=Y_{2j}=8, \forall j$;

$X_{3j}=Y_{3j}=6, \forall j$;

$X_{4j}=Y_{4j}=4, \forall j$;

$X_{5j}=Y_{5j}=2, \forall j$;

$X_{6j}=-Y_{6j}=j, \forall j$.

The correlation in this case should be $0.5428$.

If we center each variable, given that, for $1 \leq i \leq 5$, both $X_i$ and $Y_i$ have no variation, we have: $X_{ij}^*=Y_{ij}^*=0$. As for $i=6$, we get the values $X_{6j}^*=j-6.5, Y_{j6}^*=(13-j)-6.5=6.5-j$ (i.e., for the $X$'s: $-5.5, -4.5, -3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5, 4.5, 5.5$, and exactly the opposite for the $Y$'s). Since $0=-0$ and $j-6.5=-(6.5-j)$, we get: $X_{ij}^*=-Y_{ij}^* \forall i,j \rightarrow X^*=-Y^*$, implying a correlation of $-1$.