Consequence of Multicollinearity In case of perfect multicollinearity the predictor matrix is singular and therefore cannot be inverted . Under these circumstances, the ordinary least-squares estimator $\hat\beta=(\Bbb X'\Bbb X)^{-1}\Bbb X'\Bbb y$ does not exist (Wikipedia) .
I can't visualize the situation.


*

*When does the situation of perfect multicollinearity occur ?

*In case of perfect multicollinearity, why is the predictor matrix singular ?

*Under these circumstances, why  does the ordinary least-squares estimator $\hat\beta=(\Bbb X'\Bbb X)^{-1}\Bbb X'\Bbb y$  not exist ?
 A: *

*The situation of perfect multicollinearity occurs when one (or more) column(s) of the design matrix are linear combinations of the others, i.e. the set of column vectors of the matrix $\mathbf{X}$ is linearly dependent.

*As you might recall from your linear algebra course, if a matrix has size $n \times p$, with $p < n$, the maximum rank of $\mathbf{X}$ will be $p$. As the columns are not independent the rank will be lower than $p$. However, to be formal, the terms singular / non singular refer only to square matrices. In this case you should say that $\mathbf{X}$ is not full column rank.

*You can see that you need to invert the matrix $\mathbf{X}'\mathbf{X} \in \mathbb{R}^{p\times p}$, for which the property $\mathrm{rk}(\mathbf{X}) = \mathrm{rk}(\mathbf{X}'\mathbf{X})$ holds. It is clear that it can't be inverted. In fact $\mathbf{X}'\mathbf{X}$ will be nonsingular only if its rank equals $p$, but as $\mathrm{rk}(\mathbf{X}) < p$ it will be singular.
In the case of the matrix $\mathbf{X}$ having rank $p < n$, i.e. having dependent columns, the system of normal equations $\mathbf{X}'\mathbf{X} \boldsymbol{\beta} = \mathbf{X}'\mathbf{y}$ will have infinite solutions depending on $n-p$ variables. There are infinitely many vectors that solve our initial minimisation problem: $$ \min_{\boldsymbol{\beta}} \| \mathbf{y} - \mathbf{X}\boldsymbol{\beta} \|.$$ You can choose, for example, the one having minimum norm.

A: It might be helpful to add that in this situation there will be infinitely many least squares solutions.  The problem is not that a least squares solution does not exist, but rather that the least squares solution is not unique.  
The formula using $(X'X)^{-1}$ isn't applicable because the inverse matrix doesn't exist.  However, any $\beta$ that satisfies the normal equations $X'X\beta=X'y$ (there will be infinitely many of these solutions) is a least squares solution.     
