Correlated regressors but coefficient estimation is still good, why?

Question

I have read time and again that strong collinearity between regressors in OLS regression can result in inaccurate estimates for individual coefficients. To see this action, I wrote the following code in Python :

import numpy as np
import matplotlib.pyplot as plt

num_samples = 200

# Generate correlated regressors
mu = np.array([5.0, 0.0, 3.0])

# Choose covariance matrix so that x0 and x1 are strongly correlated
cov = np.array([
        [  3.40, 2.75, 0.01],
        [  2.75, 2.30, 0.02],
        [  0.01, 0.02, 2.40]
    ])

rng = np.random.default_rng()
X = rng.multivariate_normal(mu, cov, size=num_samples)

# Extract regressors
x0 = X[:,0]
x1 = X[:,1]
x2 = X[:,2]

y = 5.12*x0 + 2.34*x1 + 3.12*x2
coeff, res , rank, s = np.linalg.lstsq(X, y)

The correlation between x0, x1 and x2 is shown in scatter-plots below (Note that x0 and x1 are strongly correlated)

The OLS coefficients were :

array([5.12, 2.34, 3.12])

which is perfectly correct and there is no error in the estimation.

• Why was strong multicollinearity between x0 and x1 not an issue here ?
• What criteria can I use to determine whether or not multicollinearity is going to be an issue ?

Answer 1

That you have perfect estimates of all three coefficients should suggest to you that something strange has happened in the simulation, and I found that strangeness.

You have no random component to your $y$ variable. Therefore, except for possible numerical issues when it comes to doing arithmetic on a computer, your regression should have perfect estimates of the coefficients, zero-width confidence intervals, and $R^2=1$. There’s no randomness, so your estimates should be perfect.

However, if you tack on an error term like np.random.normal(0, 1, num_samples), you will see your estimates get worse.

Back to the original issue, the Gauss-Markov theorem gives conditions where the OLS estimator is unbiased, and the theorem makes no mention of feature correlation. However, feature correlation affects confidence interval width on the feature coefficients through the the variance-inflation factor (VIF). As features become more correlated, their confidence intervals widen. However, the estimates are unbiased, even when the correlation is high.

(You don’t even have to appeal to Gauss-Markov, but it uses a common set of assumptions, none of which involve feature correlation.)