Dealing with multicollinearity across dependent variables In an experiment, participants rated each of several musical stimuli on 5 dimensions (ratings A to E). For each of the ~3,000 trials, the independent variables were (* = critical manipulation):

*

**ensemble/group size (1 or 2)

*learning condition (1, 2, or 3)

*repetition (1 or 2)

*musical training (continuous score)

I'd like to find which rating (or combination of ratings) can best predict 'ensemble size'. The screenshot below shows the data structure, for clarity:

I analysed this data in two ways:
(1) I performed one LME for each of the 5 ratings, taking rater and stimulus ID as RFX and the remaining IVs as FFX:
    rating ~ ensembleSize + condition + rep + training + 
               (1|id_rater) + (1|stimulus)

(2) I switched the question round and, using fitglm in Matlab, created a logistic binomial model of the probability of each stimulus having been an ensembleSize=1 or 2, as a function of the 5 ratings, separately for the 3 learning conditions. So here the ratings are the IVs, and ensembleSize is the DV.
My questions:

*

*Which of the two approaches is in fact more appropriate? IV=ensembleSize, DVs=ratings, as with the LME approach? Or the logistic model where IV and DV are reversed?


*THe LME I did ignores the fact that there's high multicollinearity among the 5 ratings (A to E). I know "multicollinearity" is normally defined among IVs, but in this case my DVs are the ratings, and it seems wrong to do an LME for each if there is in fact (let's say) only one latent variable behind them all.
Perhaps a suitable correction is to compute VIFs (Variance Inflation Factors)? Or perhaps a Variance-Covariance Matrix? Or is it best to define a single multilevel model, in which the different ratings are nested within raters, themselves nested within 'stimulus' and 'learning condition'?
So far I used Matlab for this data set, but I could also continue this in R or Python, if it makes the implementation easier. Thank you for any inputs!
 A: I'm not 100% sure I understand the question, but I will do my best by interpreting what you might be interested in.
Question 1: How do I explore the multicollinearity in my data?
Firstly, figure out if your variables are multicollinear (compute VIF or effective rank). If they indeed are multicollinear, it may be of interest to know the structure of multicollinearity. Common methods include looking at correlation matrix, using PCA, FA, or ICA. These techniques can be used to perform dimensionality reduction prior to fitting. One state of the art approach is to construct a tSNE plot and color points by parameter values. If tSNE identifies distinct clusters, then explore values within each cluster, should be informative. Dimensionality reduction can also be tSNE-inspired, performing fitting on each cluster independently.
Question 2: Is multicollinearity a problem for independent variables?
If your goal is to figure out how well they collectively fit the dependent variable, then no. If the goal is to understand their individual contributions, then yes, it is a problem.
Question 3: Is multicollinearity a problem for dependent variables?
If your goal is to just get the p-values, then no, it is irrelevant. If you wish to dig deeper and understand whether two dependent variables can be predicted for the same reason, then yes, it is a problem.
For both dependent and independent variables dimensionality reduction can help identify specific components that are predictable or predictive. Note however that figuring out such components is part of exploratory analysis. So if you are interested in statistical significance, you must split your dataset in two parts: the training part, which you would use to set a hypothesis on the structure of components that dependent and independent variables will be reduced to prior to fitting, and the testing part, on which the significance tests will be carried out.
Question 4: In what ways could independent variables contribute to a dependent variable
For linear or general linear models, the the only two possible contribution types are the unique contributions of one IV and redundant (multicollinear) contributions of multiple IV. Robust disentangling between those is typically hard. Partial Correlation (Partial F-Test) can be used to estimate the size of individual contributions of each IV conditioned on all other IVs. Note however that Partial Correlation and all similar models are extremely sensitive to noise or any other imperfections in the IV. Partial Correlations of large magnitude can be used as proof that the model including the particular IV does better than the model without it. However, for realistic data the Partial F-Test is biased and comes up positive even if there is no effect at all.
On top of unique and redundant contributions, independent variables may contribute synergistically. Namely, best prediction may be achieved via a complex function of multiple independent variables that is greater than sum of its parts. The simplest example is addition of quadratic coupling terms to a linear model.
The the unique, redundant and synergistic contributions can also be studied for general non-linear cases using information-theoretic approaches, see Partial Information Decomposition. Of course, any generalization is not free, as it may require even larger data sizes and some further approximations.
Question 5: How to do analysis of confounding variables correctly
This is something I myself would really like to know. Similar to the above, you may wish to figure out if the fit of the target dependent variable (ensembleSize) is confounded by other DV you presented. Similarly to the above, Partial Correlation and similar metrics can in theory be applied to see if the dependent variables confound each other. And for similar reasons, such analysis is likely to lead to false positives, for example if the dependent variables are some transformed noisy copies of one latent variable. The only approach I have seen is to study the target dependent variable independently for each combination of other dependent variables, keeping them fixed in each analysis (as you said). Again, this dramatically reduces significance, but as far as I know there is no better way
A: Multicollinearity is not a problem.
In the world of mixed effects modeling, it's thought of as within, vs between, cluster variation, and is assessed using the intra-class correlation (ICC).
However, higher ICC means that you have less power than if the ICC was lower. This is why, if all else is equal, you would rather have fewer ratings from more participants, than more ratings from fewer participants. More participants = more power, even if the total number of ratings is equal.
