5
$\begingroup$

When two vectors $a$ and $b$ are collinear, then $a = xb$, (where $x$ is a scalar) so in linear algebra, collinearity is a narrowly and clearly defined (and binary) concept. Two vectors -- in my understanding -- are either collinear or they are not; there are no different degrees of collinearity. If $a$ and $b$ represent distinct time series or samples in econometrics, I would therefore expect to never actually find (multi)collinearity in any empirical context, because it is close to impossible that two distinct time series or samples are exact multiples of one another. We can find correlation, and I could also imagine finding variables which are -- at least approximately -- coplanar in certain contexts, but never true collinearity.

Why then is the term multicollinearity used in the context of econometrics, when what it really seems to mean is a (statistically significant) correlation between two or more explanatory variables in a multiple regression model? Is what's meant by problems of multicollinearity that the null hypothesis of no collinearity is rejected at a certain significance level, even when we know for certain that there is strictly speaking no collinearity among the explanatory variables simply by inspecting the data? Why is multiple correlation not the more accurate term to use?

I have recently encountered the terms perfect and imperfect multicollinearity and this has confused me additionally. Does someone have a rigorous understanding of this and could share it? I would very much appreciate it!

$\endgroup$
  • $\begingroup$ It's a good question. But in a comparable sense it is also the case in linear algebra that a system of linear equations $Ax=b$ has no solutions when $b$ is not in the sub-vector space spanned by the columns of $A$. Nevertheless, least squares finds solutions, but rarely "true solutions" (they would be the ones whose residuals are all zeros). $\endgroup$ – whuber Dec 11 '14 at 23:56
7
$\begingroup$

I don't think anyone worries about exact collinearity. If that was the case, $X'X$ would not be invertible. That is why full column rank of $X$ in one of the first assumptions. People worry about inexact relationships, since then there are coefficients to interpret, but they are too unreliable to be useful. The world multicollinearity is usually reserved for the latter. But one can easily become the other in the limit. Think about $\vert X'X\vert$. This determinant declines in value with increasing collinearity, tending to zero as the collinearity becomes exact. You can also consider the auxiliary regression $R^2_j$ tending to 1. Perhaps that is why we use the same term for both, though the extreme case is impossible if you have coefficients in hand.

Exact linear relationships most often arise in the context of the dummy variable trap and are easy to diagnose. The consequences of approximate linear relationship among some of the regressors in the sample at hand are indistinguishable from the consequences of inadequate variability of the regressors in a data set. Arthur Golberger joked that we should call this phenomenon "micronumerosity" instead. This is the one that people usually worry about, though it doesn't violate any of our usual assumptions.

This sort of multicollinearity, by definition, is a feature of your particular sample of data with which you're trying to fit a regression model. More or less, it means that there is insufficient information in your data to make reliable inferences about the individual parameters of the underlying (population) model, though jointly they may well be informative.

There are various sample measures that you can compute and report (VIFs or conditioning indices), to help gauge how severe this problem may be. But they're not statistical tests. Because multicollinearity is a characteristic of the sample, and not a characteristic of the population, you cannot test for it, in the same sense that it makes no sense to test that $\hat \beta = 0$, rather than $\beta = 0$. Of course, there are tests for relationships in the population that are often misinterpreted as test of multicollinearity.

$\endgroup$
  • $\begingroup$ I understand that this strict multicollinearity can arise with dummy variables, but why is the term collinearity used in the context of multiple regression models generally, where it encompasses the broader notion of critically strong correlation among predictors which in the vast majority of cases (with the exception of dummy variables) are not strictly collinear? For example, if I attempt to explain a child's income expectations as a function of income and years of education of each parent, these variables would likely be called collinear although really they are truly merely correlated. $\endgroup$ – Constantin Dec 12 '14 at 23:37
  • $\begingroup$ I am reluctant to even use the term strictly collinear because the notion of different levels, intensities, or levels of collinearity simply does not make any sense to me. $\endgroup$ – Constantin Dec 12 '14 at 23:38
  • $\begingroup$ @Constantin I edited my post to clarify. Let me know if that still doesn't make sense. $\endgroup$ – Dimitriy V. Masterov Dec 13 '14 at 0:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.