3
$\begingroup$

I was reading this when I came across the term collinearity. I tried looking up what it is but top results are related to multicollinearity.

I could find here about multicollinearity

multicollinearity refers to predictors that are correlated with other predictors in the model

It is my assumption (based on their names) that multicollinearity is a type of collinearity but not sure. Do these 2 terms differ or are they synonyms?

$\endgroup$
  • $\begingroup$ what from the wikipedia article is not clear? en.wikipedia.org/wiki/Multicollinearity#Definition $\endgroup$ – TrynnaDoStat Jan 6 '17 at 15:39
  • $\begingroup$ @TrynnaDoStat Well, nothing is unclear, but, one should avoid answering questions in comments. Example here would be that I was taking the information, condensing it, and presenting it as an answer while the comment above was made. $\endgroup$ – Carl Jan 6 '17 at 15:59
7
$\begingroup$

In statistics, the terms collinearity and multicollinearity are overlapping. Collinearity is a linear association between two explanatory variables. Multicollinearity in a multiple regression model are highly linearly related associations between two or more explanatory variables.

In case of perfect multicollinearity the design matrix $X$ has less than full rank, and therefore the moment matrix $X^{\mathsf{T}}X$ cannot be matrix inverted. Under these circumstances, for a general linear model $y = X \beta + \epsilon$, the ordinary least-squares estimator $\hat{\beta}_{OLS} = (X^{\mathsf{T}}X)^{-1}X^{\mathsf{T}}y$ does not exist.

$\endgroup$
  • $\begingroup$ The restriction of collinearity to just two variables is not standard: see en.wikipedia.org/wiki/…. "Multicollinearity" is a perfect synonym used exclusively in a multiple regression context. $\endgroup$ – whuber Jan 6 '17 at 16:47
  • 1
    $\begingroup$ @whuber I rarely disagree with you. However, the restriction of collinearity to just two variables is common usage to avoid grammatical number disagreement. It is a grammar thing, not a true statistical difference. Indeed, on the same web page you cited note that for statistical language we do differentiate between them. And, admittedly, for geometry, we would not. $\endgroup$ – Carl Jan 6 '17 at 17:19
  • 1
    $\begingroup$ Thank you for pointing out that special case in the Wikipedia article; I cede your point (+1). I respect the fact that because many different intellectual communities use and contribute to statistics, many technical terms can have varied meanings. In this particular instance, though, "multicollinearity" carries such echoes of bombastic self-importance (hey guys--to make sure people know we're serious, let's use a long, redundant, complicated technical word to describe a simple concept that already has a perfectly fine name) that I try to avoid its use. $\endgroup$ – whuber Jan 6 '17 at 17:31
  • $\begingroup$ @whuber Agreed, looking back on this, I used the term collinearity six times in Table 3, without using the term multicollinearity even once. However, the question was not about us, but about what to understand when we do see the term multicollinearity being used. $\endgroup$ – Carl Jan 6 '17 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.