Is the explained variance of a model invariant to the presence of multicollinearity?

Question

I have fitted two different binomial logistic regression models A and B. Model A contains only one predictor variable. Model B contains a different set of predictor variables, none of which is the predictor included in model A. There is a notable degree of multicollinearity between the predictors in model B. I want to compare how well the two models can account for the variation in my data.

Usually, when discussing the consequences of multicollinearity in linear regression models, most authors focus on the effect on the predictors. For example Dormann et al (2012) point out that the standard errors of collinear coefficients will be inflated, which leads to "inaccurate tests of significance for the predictors, meaning that important predictors may not be significant, even if they are truly influential" (p. 29).

However, what is less clear to me is the effect of multicollinearity on the overall performance of the model. This question asks whether multicollinearity affects the performance of the model as a classifier, with reference to the Wikipedia article on Multicollinearity which says that "multicollinearity does not reduce the predictive power or reliability of the model as a whole, at least within the sample data set". In his answer to the question, @EdM seems to confirm that multicollinearity does not affect model reliability unless it is used to predict a data set different from the one used to fit it.

My case is somewhat different because I don't want to use the models as classifiers on new data. Instead, I want to compare how well they can explain my data set. So, that answer still leaves me with the following questions:

• Is it valid to say that the explained variance of a model is invariant to the presence of multicollinearity?
• Can I use still measures such as AIC or AUROC to compare the performance of my models A and B even though the predictors of model B are strongly correlated?
• Is there a quotable reference that discusses the effect of multicollinearity on the explained variance of models and on measures such as the AIC or AUROC?

Answer 1

From the comments, it is evident that you mean some kind of predictive accuracy measurement, any of which would be a function of the predictions and the true observations.

Multicollinearity cannot affect the true observations, since they are the observations even if you don't do any modeling, so for multicollinearity to influence the predictive accuracy, multicollinearity must influence the predictions.

Let's do a simulation where we do a logistic regression on two correlated variables, regress on a transformation of those variables that retains all of the information while removing the correlation, and comparing predictions.

library(MASS)
set.seed(2022)
N <- 25
X <- MASS::mvrnorm(N, c(0, 0), matrix(c(1, 0.9, 0.9, 1), 2, 2))
y <- rbinom(N, 1, 0.5)
L_correlated <- glm(y ~ X, family = binomial)
predictions_correlated <- predict(L_correlated)
PCs <- X %*% prcomp(X)$rotation # Do PCA to remove correlation
L_uncorrelated <- glm(y ~ PCs, family = binomial) # Take all PCs to
                                                  # retain all information
predictions_uncorrelated <- predict(L_uncorrelated)
round(predictions_correlated - predictions_uncorrelated, 14)

Aside from floating point arithmetic issues out past the $14^{\text{th}}$ decimal place, the predictions are the same.

In GLMs, multicollinearity can affect interpretations, p-values, and confidence intervals, but multicollinearity does not affect the predicted values. Some people also dislike multicollinearity because they think they can drop some variables and retain most of the information while lowering the parameter count. As much as that makes sense, it is not perfect and might even be fairly problematic.