I have read time and again that strong collinearity between regressors in OLS regression can result in inaccurate estimates for individual coefficients. To see this action, I wrote the following code in Python :
import numpy as np import matplotlib.pyplot as plt num_samples = 200 # Generate correlated regressors mu = np.array([5.0, 0.0, 3.0]) # Choose covariance matrix so that x0 and x1 are strongly correlated cov = np.array([ [ 3.40, 2.75, 0.01], [ 2.75, 2.30, 0.02], [ 0.01, 0.02, 2.40] ]) rng = np.random.default_rng() X = rng.multivariate_normal(mu, cov, size=num_samples) # Extract regressors x0 = X[:,0] x1 = X[:,1] x2 = X[:,2] y = 5.12*x0 + 2.34*x1 + 3.12*x2 coeff, res , rank, s = np.linalg.lstsq(X, y)
The correlation between
x2 is shown in scatter-plots below (Note that
x1 are strongly correlated)
The OLS coefficients were :
array([5.12, 2.34, 3.12])
which is perfectly correct and there is no error in the estimation.
- Why was strong multicollinearity between
x1not an issue here ?
- What criteria can I use to determine whether or not multicollinearity is going to be an issue ?