Squared Prediction Error and PCA derivation I've been looking at the Squared Prediction Error for PCA. It is defined as follows:
$$Q = rr^\text{T}$$
Where
$$r = x(I- PP^\text{T})$$
Now it is claimed that 
$$Q = x(I- PP^\text{T})x^\text{T}$$
But I cannot see how? My reasoning is as follows:
$$Q = rr^\text{T} \\
 = x(I- PP^\text{T}) (x(I- PP^\text{T}))^\text{T} \\
 = x(I- PP^\text{T})(I- PP^\text{T})^\text{T} x^\text{T}$$
Now $(A+B)^\text{T} = A^\text{T} + B^\text{T}$ and $(AB)^\text{T} = B^\text{T}A^\text{T} $, hence 
$$(I- PP^\text{T})^\text{T}\\
= I^\text{T}- (PP^\text{T})^\text{T} \\
= I^\text{T}- PP^\text{T} \\$$ 
Therefore we find that 
$$Q = x(I- PP^\text{T})(I- PP^\text{T})x^\text{T}$$
Now clearly this is not equal to 
$$Q = x(I- PP^\text{T})x^\text{T}$$
unless some further assumptions are invoked? For example $(I- PP^\text{T})(I- PP^\text{T})$ can be expanded to give
$$(I- PP^\text{T})(I- PP^\text{T}) \\
 = II -2PP^\text{T} +PP^\text{T}PP^\text{T} \\
 = I -2PP^\text{T} +PP^\text{T}PP^\text{T} \\
$$
If $P^\text{T}P = I$ then this would reduce to $I - PP^\text{T}$. But if $P^\text{T}P = I$ then surely $PP^\text{T} = I$ in which case 
$$Q = x(I- PP^\text{T})x^\text{T} = 0 \:\: \forall \:\: x$$
And hence $PP^\text{T} \neq I$, $P^\text{T}P \neq I$ and $Q \neq x(I- PP^\text{T})x^\text{T}$. Am I missing something? 
 A: Apologize for not explaining well previously
Let $z_n\in\mathbb{R}^{M}$ be the hidden/latent variables obtained by projecting the observed variables as the following,
\begin{equation}
z_n = P^Tx_n
\end{equation}
where $x_n\in\mathbb{R}^D$ is a vector of observed variables and $P\in\mathbb{R}^{D\times M}$ is of $M$ orthonormal eigenvectors of the covariance of $x_n$.
The optimal reconstruction of $x_n$ from $z_n$ in the least squares sense is given by,
\begin{equation}
\hat{x}_n = Pz_n
\end{equation}
where $\hat{x}_n$ is the reconstructed observation.
If you replace $z_n$ in the above equation, you will get $\hat{x}_n$ in terms of the original observation as the following,
\begin{equation}
\hat{x}_n = PP^Tx_n
\end{equation}
The reconstruction residuals are given by,
\begin{equation}
r_n = x_n - \hat{x}_n = x_n - PP^Tx_n = (I - PP^T)x_n
\end{equation}
where $r_n$ are the reconstruction residuals or errors.
Therefore, SPE should be expressed as,
\begin{equation}
SPE = Q = r_n^Tr_n =  x_n^T(I - PP^T)^T(I - PP^T)x_n
\end{equation}
Further, as $(I - PP^T)^T$ is symmetric, we can rewrite the above equation as, 
\begin{equation}
Q =  x_n^T(I - PP^T)(I - PP^T)x_n
\end{equation}
and eventually as,
\begin{equation}
Q =  x_n^T(I - PP^T - PP^T + PP^TPP^T)x_n
\end{equation}
where, $P^TP = I$ as $P$ is an orthonormal matrix, which allows further simplification of the following form,
\begin{equation}
Q =  x_n^T(I - PP^T)x_n
\end{equation}
You would probably have figured this out. Hopefully, this helps others!
