# Two ways to get rid of multicollinearity

I have a couple of questions concerning multicollinearity in a linear regression model $$Y=X \beta + \epsilon$$.

If the design matrix presents some multicollinearity i.e. $$\det(X^TX) \approx 0$$, we can fix it by replacing it by a design matrix $$Z$$ of column that form an orthogonal basis of $$Im(X)$$. To do that, it is said to consider $$X^TX$$ and its diagonalised form : $$X^T X=U D U^T$$ and to take $$Z=XU$$. However, I don't see how does that solve multicollinearity because $$(XU)^T XU=D$$ so that $$\det(X^TX)=\det(Z^TZ)=\det((XU)^TXU)\approx 0$$. So we did not solve multicollinearity ?

Furthermore, I understand that this manipulation make us lose interpretation, instead of having a column for age, height and weight for example, we'll have a matrix with columns like 0.4 age + 0.1 height + 0.5 weight which does not allow a nice interpretation. One way to solve this problem without losing interpretation is to use ridge regression. To do so, we start by standardising the design matrix, writing first as $$X=(\textbf1 \quad W)$$, $$\beta=(\beta_0 \quad \gamma)$$ (we separate the intercept (the column with 1 in each entry) from the other columns). It is said in my course "we then rescale the covariates by defining $$Z_j=\frac {\sqrt n}{sd(W_j)} (W_j - \textbf1 \overline{W_j})$$ so that coefficients now have common scale". As an explanation, our teacher said that in the initial design matrix $$X$$, if we change the units of measurement of one of the covariates (from meters to miles for example), then the magnitude of the corresponding coefficient in $$\beta$$ will change too (I agree) and he said we won't have this problem when using the $$Z_j$$. I think this example does not illustrate well what he meant by "coefficients now have common scale". Can someone come up with a better explanation ?

• The determinant of $X^\prime X$ has nothing to do with multicollinearity unless it is exactly zero. In particular, the columns of $D$ as as orthogonal (non-collinear) as one can possibly get, no matter what the determinant of $D$ might be. It's unclear what you're trying to ask in the second part of this post. Each of the $Z_j$ is constructed to have a unit standard deviation, which constitutes its scale. What further explanation are you looking for?
– whuber
Nov 30, 2022 at 16:20