5
$\begingroup$

I was reading this when I came across the term collinearity. I tried looking up what it is but top results are related to multicollinearity.

I could find here about multicollinearity

multicollinearity refers to predictors that are correlated with other predictors in the model

It is my assumption (based on their names) that multicollinearity is a type of collinearity but not sure. Do these 2 terms differ or are they synonyms?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
13
$\begingroup$

In statistics, the terms collinearity and multicollinearity are overlapping. Collinearity is a linear association between two explanatory variables. Multicollinearity in a multiple regression model are highly linearly related associations between two or more explanatory variables.

In case of perfect multicollinearity the design matrix $X$ has less than full rank, and therefore the moment matrix $X^{\mathsf{T}}X$ cannot be matrix inverted. Under these circumstances, for a general linear model $y = X \beta + \epsilon$, the ordinary least-squares estimator $\hat{\beta}_{OLS} = (X^{\mathsf{T}}X)^{-1}X^{\mathsf{T}}y$ does not exist.

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ The restriction of collinearity to just two variables is not standard: see en.wikipedia.org/wiki/…. "Multicollinearity" is a perfect synonym used exclusively in a multiple regression context. $\endgroup$
    – whuber ♦
    Jan 6, 2017 at 16:47
  • 2
    $\begingroup$ @whuber I rarely disagree with you. However, the restriction of collinearity to just two variables is common usage to avoid grammatical number disagreement. It is a grammar thing, not a true statistical difference. Indeed, on the same web page you cited note that for statistical language we do differentiate between them. And, admittedly, for geometry, we would not. $\endgroup$
    – Carl
    Jan 6, 2017 at 17:19
  • 4
    $\begingroup$ Thank you for pointing out that special case in the Wikipedia article; I cede your point (+1). I respect the fact that because many different intellectual communities use and contribute to statistics, many technical terms can have varied meanings. In this particular instance, though, "multicollinearity" carries such echoes of bombastic self-importance (hey guys--to make sure people know we're serious, let's use a long, redundant, complicated technical word to describe a simple concept that already has a perfectly fine name) that I try to avoid its use. $\endgroup$
    – whuber ♦
    Jan 6, 2017 at 17:31
  • $\begingroup$ @whuber Agreed, looking back on this, I used the term collinearity six times in Table 3, without using the term multicollinearity even once. However, the question was not about us, but about what to understand when we do see the term multicollinearity being used. $\endgroup$
    – Carl
    Jan 6, 2017 at 18:48
0
$\begingroup$

There are indeed slight inconsistencies in the usage of the term, depending who you ask. The most common distinction I've seen (and I tend to use), is that we have collinearity if $\det(X^T X)=0$, and multicollinearity if $\det(X^T X)\approx 0$. The latter obviously includes the former, which is why we also say "perfect multicollinearity" for "collinearity".

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Could you cite some example references please? $\endgroup$
    – Nick Cox
    Feb 23, 2022 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.