Why do sources claim that linear regression assumes little to no multicollinearity, among other "assumptions"?

Question

Many sources (here is one of many https://towardsdatascience.com/assumptions-of-linear-regression-algorithm-ed9ea32224e1) state that linear regression assumes there is little to no multicollinearity between the independent variables. I don't understand why this is the case.

Linear regression assumes the model looks like the following: $$ Y = X\beta + \epsilon $$

There is nothing about this model that assumes little to no multicollinearity. Now if you were to approximate this model with least squares, then yes, if there were to be perfect multicollinearity, then your least squares estimators are no longer unique, or if there were strong multicollinearity, but not perfect, then the condition number of $(X^TX$ will be large, meaning that $\hat{\beta}$ is unstable, but even then there is no such assumption about "little or no multicollinearity."

So why do sources claim that linear regression assumes multicollinearity?

In my opinion, the only thing that linear regression assumes is that $Y = X\beta + \epsilon$, i.e., $Y$ varies linearly with $X$ subject to some unobservable $\epsilon$. That's it. No other assumptions. A simple Google search for "linear regression assumptions" will produce numerous links that the assumptions of linear regression as "normal distribution of errors," "little to no multicollinearity," "Homoscedasticity," among others. I just don't understand how these are "assumptions." They're more like implicit requirements for the underlying data in order for linear regression to produce a good fit.

It might be correct to say that in order to evaluate a certain linear regression method (e.g., least squares), we should assume x, y, z, but saying that linear regression assumes x, y, z seems misleading.

Here's another link that stood out to me https://medium.com/@dhiraj8899/top-5-assumptions-for-linear-regression-dd8d10bb0039. In the beginning, it states "For a linear regression algorithm to work properly, it has to pass at least the following five assumptions." I don't know if this is poor wording, or if I'm being anal, or what, but assumptions aren't what's going to make an algorithm work properly. Making an assumption has nothing to do with your algorithm working.

Answer 1

There is nothing about the model that assumes anything about co-linearity. The first link is misleading (and the second is just wrong).

Co-linearity is an issue in interpretation, though. Further down in that first linked document it says

The stronger the correlation, the more difficult it is to change one feature without changing another. It becomes difficult for the model to estimate the relationship between each feature and the target independently because the features tend to change in unison.

That's still not quite true, but it's pointing in an important direction. Suppose you want a model where the coefficient of $X$ describes the effect of changing $X$ either (less ambitiously) on your prediction or (more ambitiously) on the actual outcome variable $Y$.

Before we understood causal inference, the simplest way to think about this was that a designed experiment gave you uncorrelated predictors, and the coefficients were effects of changing those predictors, and if you had observational data where the predictors were basically uncorrelated, the coefficients would still have those interpretations.

This wasn't quite right, because it was thinking in terms of correlations, which aren't the right causal quantities (eg they are symmetric, which cause and effect aren't), but it was a helpful way to think if you didn't have better tools. And it was true to say that interpreting the coefficients is hard if the correlations are not small.

We have better tools now: if you want to set up a model where a coefficient has a causal interpretation, we know the criteria it will have to satisfy (and it probably won't satisfy them).

Also, generalisability to other distributions of $X$ can tend to be worse with co-linearity: if the 'training' data do not vary much in one direction, and your new data are different along that one direction, it would be optimistic to expect the model to generalise well. So, if you have co-linearity you'd want to know whether it was a property just of your particular data set or of the whole phenomenon you're studying.

In summary: no, it isn't an assumption of the model, but it is (or at least used to be) an assumption of the people doing the modelling, for not entirely bad reasons.

Answer 2

Forgive me, but I must say that "source" is not someone who speaks of something, and "reliable source" is not what appears at the top in a Google search.

You should better look at Arthur Goldberger, A Course in Econometrics. The assumptions of "classical" regression are:

1. $E(y)=X\beta$
2. $V(y)=\sigma^2 I$
3. $X$ non stochastic
4. $X$ full rank

The assumptions of "neoclassical" regression (with $X$ random) are:

1. $E(y|X)=X\beta$
2. $V(y|X)=\sigma^2 I$
3. $X$ stochastic
4. $X$ full rank

Period. As to multicollinearity there is a famous chaper in that book, summarised by Bruce Hansen, Econometrics, pages 124-126: it was just a concern overemphasized by some earlier textbooks.