Does multicollinearity among control variables matter?

Question

I am conducting a regression analysis between $X$ and $Y$, where $X$ is the main independent variable. However, I want to control for several variables that are related to $Y$. For example, my dependent variable is loan default and $X$ is a unique independent variable, but I also want to control for loan characteristics such as interest, maturity, etc. Note, I have only one independent variable. Does multicollinearity matter between control variables or is it only for independent variables?

Answer 1

NO

(Assume ordinary least squares linear regression, though I expect a similar argument to work for other models.)

If the control variables are not multicollinear with our variables of interest, they do not affect the inference on the parameters on those variables of interest.

Consider the formula for variance inflation factor. It is this inflation of the variance of the coefficient estimates that we would prefer to avoid. However, if the rest of the variables are not predictive of the variables of interest, there is no variance inflation.

(I remember first discovering this when I thought I can wrestle out some additional power by running PCA on my control variables and retaining all of the PC, thinking that this was a trick to remove the covariance but retain all of the information, only to find my tests of the other variable completely unaffected.)

Answer 2

Below, I will presume that we use ordinary least squares linear regression.

If you have multicollinearity only among the control variables, let's call them $Z_i, i=1, \ldots, k$, but the treatment variable $X$ is not part of the subspace spanned by the control variables, then you can use linear regression to obtain the effect of the treatment $X$ on the outcome.

Here is a figure which might help with the geometric intuition:

We consider the situation of a data set with only two data points, but this is easily generalizable to higher dimensions. Also, we presume that $k=2$, i.e. we have two control variables, $Z_1$ and $Z_2$. The vectors in the figure symbolize the columns of the design matrix, so the red vector $x$ is the column vector of the treatment $X$, the purple vector $y$ is the column of the outcome $Y$, and $z_1$ and $z_2$ are the columns of the covariates. As required, $z_1$ and $z_2$ are collinear and $x$ is not part of the subspace spanned by $z_1$ or $z_2$.
Clearly, to obtain a linear combination of the outcome $y$, we need to obtain the black cross by a linear combination $\lambda_1 z_1 + \lambda_2 z_2$ and then add a multiple of $x$ (the dotted red vector). For the linear combination $\lambda_1 z_1 + \lambda_2 z_2$ there are infinitely many possibilities. The linear regression algorithm will return one of them and, maybe, a collinearity warning. However, we are not interested in the coefficients of the covariates, but in that of the treatment $x$, and that one is clearly unique, as long as $x$ is not part of $\operatorname{span}(z_1)$.

If, on the other hand, $x$ would be collinear with $z_1$ and $z_2$, i.e. $x\in\operatorname{span}(z_1)$, the situation would be as follows:

Now, the best linear approximation of the outcome with the collinear vectors $x, z_1, z_2$ would be the orthogonal projection $Py$ of $y$ to $\operatorname{span}(z_1)$. Again, we have infinitely many solutions, but this, time, unfortunately, the coefficient of $x$ is not unique anymore, either. Thus, in this case, the real causal effect of $X$ on $Y$ cannot be identified from observational data and we would have to resort to interventions on the treatment.