Since a projection matrix is idempotent, symmetric and square, why isn't it just the identity matrix? I was working on a question on projection matrix. Since, projection matrix is idempotent, symmetric and square matrix, it must always be equal to $I$ (Identity matrix). This can be shown by multiplying the inverse of projection matrix on both the sides. If it is equal to $I$, then I do not understand the point of using it. Can anyone please explain?
 A: There are a couple excellent answers already, but I'd like to add one more perspective (hinted at in @Antoni's answer).
An idempotent matrix satisfies the matrix equation
$$ X^2 = X $$
or
$$ X^2 - X = 0 $$
Which we can factor
$$ X(X - I) = 0 $$
This means there are relatively few possibilities for the minimal polynomial of $X$
The Minimal Polynomial is $p(x) = x$:
In this case $X = 0$ immediately, we have the zero matrix.  Not very interesting.
The Minimal Polynomial is $p(x) = x - 1$:
In this case $X - I = 0$, so $X = I$.  This is the case you are thinking of.
The Minimal Polynomial is $p(x) = x(x - 1)$:
This is the interesting "in between" case.
Because the minimal polynomial splits into linear factors, the matrix $X$ is diagonalizable, with only $0$ and $1$'s on the resulting diagonal.
Another way to look at this: $X$ has a full linearly independent set of eigenvectors, say
$$ e_1, e_2, \ldots, e_{k_1}, f_1, f_2, \ldots, f_{k_2} $$
With $k_1 + k_2 = n$.  Some of the eigenvectors have an associated eigenvalue of zero
$$ X e_1 = 0, X e_2 = 0, \ldots, X e_{k_1} = 0$$
and the rest have an eigenvalue of one
$$ X f_1 = f_1, X f_2 = f_2, \ldots, X f_{k_2} = f_{k_2}$$
Now a geometric picture emerges, and it justifies the name projection matrix.  If we look at the subspace spanned by the $f$'s, then the image of any vector is in this subspace
$$ X (a_1 e_1 + \cdots + a_{k_1} e_{k_1} + b_1 f_1 + \cdots + b_{k_2} f_{k_2}) = b_1 f_1 + \cdots + b_{k_2} f_{k_2} $$
and, by the same calculation, the mapping restricted to this subspace is indeed the identity.  You see, the map is really a projection into the subspace spanned by the $f$'s.
A: An idempotent matrix https://en.wikipedia.org/wiki/Idempotent_matrix (see also https://en.wikipedia.org/wiki/Projection_(linear_algebra) ) is a matrix which equals its square.  So $P$ being idempotent means that $P^2 = P$.  The identity matrix is idempotent, but is not the only such matrix.
Projection matrices need not be symmetric, as the the 2 by 2 matrix whose rows are both $[0,1]$, which is idempotent, demonstrates. This provides a counterexample to your claim.  By (pre-)multiplying both dies of $P^2 = P$, you have lost solutions.  
Edit: In response to the comment by @Antoni Parellada , the OLS projection matrix $X(X^TX)^{-1}X^T$ is idempotent and symmetric, but in general it is not equal to the identity matrix, although it is possible that it could equal the identity matrix in some special (unusual) case.
