# Computing condition number for OLS

In https://www.statsmodels.org/0.8.0/examples/notebooks/generated/ols.html, if you scroll down to the section "Multicollinearity" (bolded), it shows that the condition number of $$X$$ is $$4.86e+09$$. Then right below that, it says

Condition number One way to assess multicollinearity is to compute the condition number. Values over 20 are worrisome (see Greene 4.9). The first step is to normalize the independent variables to have unit length

and then they compute another condition number using

norm_x = X.values
for i, name in enumerate(X):
if name == "const":
continue
norm_x[:,i] = X[name]/np.linalg.norm(X[name])
norm_xtx = np.dot(norm_x.T,norm_x)

eigs = np.linalg.eigvals(norm_xtx)
condition_number = np.sqrt(eigs.max() / eigs.min())
print(condition_number)


and the result comes out to be 56240.8714071 .

I'm confused about several things here.

(1) Are we more interested in the condition number of $$X$$ or $$X^TX$$ when assessing the conditioning of the system for OLS? The linear system of equations is $$X \beta = y$$, but the normal equations are $$X^TX\beta = X^Ty$$, which is also a linear system of equations. I'm assuming this depends on the method since some methods for solving OLS don't even form the normal equations. I think statsmodels in Python uses SVD, which just performs the decomposition for $$X$$, so what's the point of computing the condition number for a normalized version of $$X^TX$$ here?

(2) Why are they normalizing the independent variables before computing the condition number?

• Does this answer your question? Relationship between SVD and PCA. How to use SVD to perform PCA? Your question is essentially the close relationship between the covariance matrix $X^TX$ used in PCA and the SVD discussed on that page. Note the part of the answer about the importance of centering in this context, the first step in normalization.
– EdM
May 7, 2021 at 14:39
• For better advice (than the stuff you quote) on detecting and diagnosing collinearity, see our thread at stats.stackexchange.com/questions/173665.
– whuber
May 7, 2021 at 15:00
• @EdM I don't think so because I'm referring to condition number specifically, and the condition number for $X$ is not equivalent to $X^TX$.
– 24n8
May 7, 2021 at 16:19
• @whuber Yes, I think normally I would use VIF, but I think condition number as some use when it comes to just a quick check for multicollinearity, and perhaps use that quick check to decide if we should use OLS or something like Ridge. With VIF we have to separately regress $p$ variables onto the other $p - 1$ variables, so it's not as of a quick check
– 24n8
May 7, 2021 at 16:23
• The condition number of $X^\prime X$ is the square of the condition number of $X,$ so they are equivalent: they're just different ways of expressing the same property of $X.$ In the link I gave, the formula for VIF does not require regressing all variables on all others: it needs only the diagonal elements of the the inverse of $X^\prime X.$
– whuber
May 7, 2021 at 16:35