In several posts, such as Is centering a valid solution for multicollinearity?, it states that centering doesn't solve multicollinearity because "it's a linear transformation."

I just made up a simple example. Consider $$ X = \begin{bmatrix} 1 & 2\\ 0 & 0 \end{bmatrix} $$ Clearly, the columns are linear dependent here, hence we have multicollinearity. If you were to center $X$, you get $$ X = \begin{bmatrix} 0.5 & 1\\ 0 & -1 \end{bmatrix} $$ Now the columns are no longer linear dependent, and the data matrix is full column rank. So doesn't centering, at times, solve multicollinearity?

  • 5
    $\begingroup$ You misinterpret the meaning of "it's a linear transformation:" in the context, the point was it's a linear transformation of a single variable at a time. Any degree of multicollinearity, short of perfect linear dependence, always can be removed with a suitable linear transformation of all the variables simultaneously. Your example is incorrect: the centered model matrix is $\pmatrix{1/2&1\\-1/2&-1}$ which is still singular. $\endgroup$
    – whuber
    Jul 10, 2020 at 21:14
  • $\begingroup$ @whuber Are you saying that if there's perfect multicollinearity, then suitable linear transformations of all variables can no longer remove the multicollinearity? Also, if it was the linear transformation of a single variable at a time, wouldn't the matrix actually be (assuming we center the second variable only) $$ \begin{bmatrix} 1 & 1 \\ 0 & =1 \end{bmatrix} $$? In which case would be full rank. $\endgroup$
    – roulette01
    Jul 10, 2020 at 21:39
  • 4
    $\begingroup$ Centering is allowable only when the range of $X$ includes the constant vector. (That's because centering is a linear combination of a column vector and the constant vector and that will leave the model unaffected only when the constant vector is already in the model.) That's not the case with your $X,$ so it's an invalid example. As far as the linear transformation claim goes, that's basic linear algebra: a linear transformation can never increase the dimension of a subspace. Perfect collinearity means the image of $X$ is a subspace of smaller dimension than the number of variables, QED. $\endgroup$
    – whuber
    Jul 10, 2020 at 21:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.