# Correlated regressors but coefficient estimation is still good, why?

I have read time and again that strong collinearity between regressors in OLS regression can result in inaccurate estimates for individual coefficients. To see this action, I wrote the following code in Python :

import numpy as np
import matplotlib.pyplot as plt

num_samples = 200

# Generate correlated regressors
mu = np.array([5.0, 0.0, 3.0])

# Choose covariance matrix so that x0 and x1 are strongly correlated
cov = np.array([
[  3.40, 2.75, 0.01],
[  2.75, 2.30, 0.02],
[  0.01, 0.02, 2.40]
])

rng = np.random.default_rng()
X = rng.multivariate_normal(mu, cov, size=num_samples)

# Extract regressors
x0 = X[:,0]
x1 = X[:,1]
x2 = X[:,2]

y = 5.12*x0 + 2.34*x1 + 3.12*x2
coeff, res , rank, s = np.linalg.lstsq(X, y)


The correlation between x0, x1 and x2 is shown in scatter-plots below (Note that x0 and x1 are strongly correlated)   The OLS coefficients were :

array([5.12, 2.34, 3.12])


which is perfectly correct and there is no error in the estimation.

• Why was strong multicollinearity between x0 and x1 not an issue here ?
• What criteria can I use to determine whether or not multicollinearity is going to be an issue ?
• The error of coefficients is relatively larger for correlated regressors, but if the noise is small and/or the sample size large then this may not be a problem. Dec 28, 2021 at 11:52
• @BigBendRegion : The true values are 5.12, 2.34 and 3.12 (see second last line in code) Dec 28, 2021 at 14:21
• The true values are in mu, as @BigBendRegion noted. The penultimate line displays the estimates of those values. The estimates differ from the true values: there is indeed estimation error.
– whuber
Dec 28, 2021 at 17:46
• @whuber : Please correct me if I am wrong. Values in mu are the mean values of x0, x1 and x2, while the OLS coefficients computed 5.12, 2.34, 3.12 are the estimates of coefficients c for the equation y=X@c. Dec 29, 2021 at 6:06
• That looks right--I misread your code. The problem with your example is that you have introduced no error in y: it is a perfect linear combination of the columns of X. No amount of collinearity in X will create difficulties in such a circumstance.
– whuber
Dec 29, 2021 at 16:39

You have no random component to your $$y$$ variable. Therefore, except for possible numerical issues when it comes to doing arithmetic on a computer, your regression should have perfect estimates of the coefficients, zero-width confidence intervals, and $$R^2=1$$. There’s no randomness, so your estimates should be perfect.
However, if you tack on an error term like np.random.normal(0, 1, num_samples), you will see your estimates get worse.