Looking at how covariance/correlation between variables differs in two groups?

Question

I have a few hundred variables representing different biomarkers. These variables have been measured in both cases and controls. The underlying units of measurement are not important, so I have standardized all variables (subtract mean, divide by SD). The standardization was done separately in cases and in controls, since the two groups are expected to have different means for some of the measurements. Within a group, most variables followed a normal distribution, a few that did not were log-transformed to get them closer to normal before standardizing (those variables can be left out if necessary).

Some variables will correlate with each-other in both controls and cases. Other variables may differ in the patterns of correlation they show with each-other in cases vs. controls. Correlations present in both cases and controls can be thought of as noise that I want to remove, so I can get only the case-specific correlations.

My end goal is to look for sets of markers that tend to group together in cases (using a method like factor analysis or PCA).

My original idea was to subtract the covariance matrix for controls from the covariance matrix for cases. But as pointed out in the comments on this question, that won't work because it has the potential to produce negative variances in some of the cells. (Or in this case, since the covariance matrix for standardized variables is a correlation matrix, subtracting would produce 0s on all the diagonals.)

Is there a better approach to take to look at the question "Are there groups of variables that tend to be correlated with each-other specifically in cases, after accounting for any patterns that cases share with controls"?

Answer 1

Suppose the control data matrix is $Z$ and for tidyness suppose that each column has mean zero. Let the control correlation matrix is $R$ and assume it is not singular. Let $R^{-1/2}$ be a square root of the inverse of $R$. The matrix $R^{-1/2}Z$ has uncorrelated columns; its correlation matrix is the identity, $R^{-1/2}RR^{-1/2}$

Now let $Y$ be the case data matrix, also with each column centred at zero. The matrix $R^{-1/2}Y$ would have uncorrelated columns if the correlations were the same, so its correlation matrix is a summary of the 'extra' correlations in the cases.

Note that if we write $S$ for the correlation matrix of $Y$, $R^{-1/2}SR^{-1/2}$ is symmetric with non-negative eigenvalues and so is suitable for PCA. Or you could just do singular-value decomposition on $R^{-1/2}Y$.

(I think I'd actually prefer doing this with covariances rather than correlations, so that differences in variance between the two groups don't look like differences in correlation)

Answer 2

A raw, untested thought. Maybe try this. It seems to me that your interest requires to do factor analysis first.

Do FA on control sample matrix. (I would perhaps recommend to do all the analysis on correlations rather than on covariances because differences in variances between case and control samples may be of lesser importance than differences in correlational patterns.). So, factor correlations of the control sample and arrive at a satisfying interpretable solution (you will need a rotation to interpret the common factors.).

Get the matrix of restored correlations.

Subtract those from the correlations of the case sample. Perform FA of the obtained "residual" or remnant correlations. You essentially "wash out" the factor (= correlational) pattern of controls from the cases, and factor analyse what's left. But what's left is a mixture of the pattern unique for the cases and the noise (nuisance factors) unique of the case sample.

One tricky moment of the subtraction is whether to subtract the diagonal elements too or to leave 1s there. One has to think over this. My immediate, inconclusive thought is to leave 1s. So that in the 2nd FA we do not restrict communalities by the uniquenesses from the 1st one.

A modified approach may be to do the 1st FA on the pooled correlation matrix instead of the correlation matrix of controls. Less contrasting results are expected, because now we reason the "common pattern" as extracted from both sides rather than from one specific side.

Answer 3

A slightly different approach I believe would work here is

• Transform your correlation matrices using the inverse hyperbolic tangent (artanh) function (the Fisher transformation)
• Calculate the difference, $\Delta$, between your two artanh transformed correlation matrices.
• On the artanh scale, the standard error of each correlation coefficient is $\frac{1}{N-3}$. The standard error of the $\Delta$ is therefore $\sqrt{se_1^2 + se_2^2}$.
• Divide $\Delta$ by its standard error to obtain z-statistics.
• Any z statistic greater than $2$ in absolute value is statistically significant at $p < .05$ (although adjustment for multiple comparisons is probably appropriate here).