I have read time and again that strong collinearity between regressors in OLS regression can result in inaccurate estimates for individual coefficients. To see this action, I wrote the following code in Python :
import numpy as np
import matplotlib.pyplot as plt
num_samples = 200
# Generate correlated regressors
mu = np.array([5.0, 0.0, 3.0])
# Choose covariance matrix so that x0 and x1 are strongly correlated
cov = np.array([
[ 3.40, 2.75, 0.01],
[ 2.75, 2.30, 0.02],
[ 0.01, 0.02, 2.40]
])
rng = np.random.default_rng()
X = rng.multivariate_normal(mu, cov, size=num_samples)
# Extract regressors
x0 = X[:,0]
x1 = X[:,1]
x2 = X[:,2]
y = 5.12*x0 + 2.34*x1 + 3.12*x2
coeff, res , rank, s = np.linalg.lstsq(X, y)
The correlation between x0
, x1
and x2
is shown in scatter-plots below (Note that x0
and x1
are strongly correlated)
The OLS coefficients were :
array([5.12, 2.34, 3.12])
which is perfectly correct and there is no error in the estimation.
- Why was strong multicollinearity between
x0
andx1
not an issue here ? - What criteria can I use to determine whether or not multicollinearity is going to be an issue ?
mu
, as @BigBendRegion noted. The penultimate line displays the estimates of those values. The estimates differ from the true values: there is indeed estimation error. $\endgroup$mu
are the mean values ofx0
,x1
andx2
, while the OLS coefficients computed5.12, 2.34, 3.12
are the estimates of coefficientsc
for the equation y=X@c. $\endgroup$y
: it is a perfect linear combination of the columns ofX
. No amount of collinearity inX
will create difficulties in such a circumstance. $\endgroup$