# Multicollinearity in simple linear regression

If there's perfect or near multicollinearity problem in a simple linear regression $$y_i = a + b x_i + u_i$$, what characteristics does $$x_i$$ have?

I think if there's perfect multicollinearity, it means that $$x_i$$ is simply a constant.

But if there's near multicollinearity, what happens to $$x_i$$?

• The one of the assumption of the bivariate regression is that there should be some variability in X (ie non constant). If X is constant then you can not identify your parameters ie a and b. Commented Jun 11, 2019 at 6:47
• What does multicolinearity mean in the case of simple regression? Commented Jun 11, 2019 at 7:29

$$\newcommand{\e}{\varepsilon}\newcommand{\one}{\mathbf 1}\newcommand{\x}{\mathbf x}$$If we're doing simple linear regression we can view this as $$y = X\beta + \e$$ with $$X = \left[\begin{array}{cc}1 & x_1 \\ \vdots & \vdots \\ 1 & x_n\end{array}\right] = [\one \; \x] \in \mathbb R^{n\times p}.$$ This already makes it clear that perfect multicollinearity happens when $$\x \in \text{span}(\one)$$, i.e. $$\x$$ is constant, but we can say more than this.
To get $$\hat\beta$$ we need $$X^TX = \left[\begin{array}{cc}n & n\bar x \\ n\bar x & \x^T\x\end{array}\right]$$ which we need to be invertible, which happens if and only if $$\det X^TX \neq 0$$. We have $$\det X^TX = n\x^T\x - n^2\bar x^2 = n(\x^T\x - n\bar x^2)$$ and it's well-known that $$\x^T\x - n\bar x^2 = \sum_{i=1}^n (x_i - \bar x)^2$$ so $$\det X^TX > 0\iff\sum_{i=1}^n (x_i - \bar x)^2 > 0.$$ If $$\x \in \text{span}(\one)$$ then $$\det X^TX = 0$$, and this also shows that the smaller the sample variance of $$\x$$ is, the more ill-conditioned this regression is.
Furthermore, under the assumption that $$\text E(\e) = \mathbf0$$ and $$\text E(\e\e^T) = \sigma^2 I$$ we know $$\text{Var}(\hat\beta) = \sigma^2 (X^TX)^{-1}$$ and $$X^TX$$ is 2x2 so we can work out that $$(X^TX)^{-1} = \frac{1}{\det X^TX} \left[\begin{array}{cc}\x^T\x & -n\bar x \\ -n\bar x & n\end{array}\right]$$ which shows that the standard errors will also increase as $$\x$$ gets closer and closer to being in $$\text{span}(\one)$$, as measured by the sample variance of $$\x$$. This is just a consequence of the usual result that multicollinearity inflates standard errors but it's nice that we can so explicitly see how it does it here.
• @Dayeon yeah $X$ has perfect multicollinearity if and only if $\mathbf x$ has no variation. If there is any variation in $\mathbf x$ then there isn’t perfect multicollinearity and $X^TX$ is invertible. The more variation there is in $\mathbf x$ the less multicollinearity there is and the standard errors reflect that