In PCA, what is the connection between explained variance and squared error? For an observations matrix $X$, I use PCA to reduce the dimension of the data to $L$. I know that in this case, PCA is guaranteed to minimise the mean reconstruction error. In Wikipedia notation, something like $$\|X-T_LW_L^T\|^2_2$$ where $W$ are the loadings, $T$ are the scores, and the subscript $L$ indicates that only $L$ components are used. 
I can also ask how much of the total variance is explained by these $L$ components.
My question is, is there a mathematical relation between the two? Assuming I have only the squared error and the covariance matrix for $X$, can I compute the explained variance?
 A: If your $\newcommand{\X}{\mathbf X}\X$ is a $n\times d$ matrix with column means subtracted and $\newcommand{\W}{\mathbf W}\W_L$ is a $d \times L$ matrix consisting of $L$ principal directions (eigenvectors of the covariance matrix), then reconstruction error is given by $$\mathrm{Error}^2 = \|\X - \X\W_L^\vphantom{\top}\W_L^\top\|^2.$$ Note that this is not a mean squared error, it is the sum of squared errors. Here $\X\W_L$ is what you called $\mathbf T_L$ in your question and $\|\cdot\|$ denotes Frobenius norm.
The proportion of explained variance in PCA can be defined as $$\text{Proportion of explained variance} = \frac{\|\X\W_L\|^2}{\|\X\|^2},$$ i.e. it is a ratio of scores' sum of squares to the overall sum of squares (or equivalently, a ratio of scores' total variance to the overall total variance).
To see the connection between them, we need the following identity: $$\|\X\|^2 = \|\X - \X\W_L^\vphantom{\top}\W_L^\top\|^2 + \|\X\W_L^\vphantom{\top}\W_L^\top\|^2 = \|\X - \X\W_L^\vphantom{\top}\W_L^\top\|^2 + \|\X\W_L\|^2.$$ It might look mysterious, but is actually nothing else than Pythagoras theorem; see an informal explanation in my answer to Making sense of principal component analysis, eigenvectors & eigenvalues and a formal explanation in my answer to PCA objective function: what is the connection between maximizing variance and minimizing error?
We now see that $$\text{Proportion of explained variance} = 1-\frac{ \text{Error}^2}{\|\X\|^2}.$$
If you know the covariance matrix $\mathbf C = \frac{1}{n-1}\X^\top \X$, then you can compute the total variance $\operatorname{tr}\mathbf C$. The squared norm $\|\X\|^2$ is then given by the total variance multiplied by $n-1$. So we finally obtain
$$\text{Proportion of explained variance} = 1-\frac{\text{Error}^2}{(n-1)\operatorname{tr}\mathbf C}.$$
A: Let
$
W= \left[ \begin{matrix}W_1 & W_2\end{matrix} \right]$ and
$T= \left[ \begin{matrix}T_1 & T_2\end{matrix} \right]$
be the partitioned $W\in\mathbb{R}^{M\times M}$ and $T\in\mathbb{R}^{N\times M}$ matrices,
where $W_1$ and $T_1$ are the submatrices containing columns $1\dots L$
of $W$ and $T$ respectively. The data matrix can be written in terms of these submatrices as
$$
X = TW^\top = T_1 W_1^\top + T_2 W_2^\top
$$
which implies $T_1=XW_1$ and $T_2=XW_2$. On the other hand the reconstructed data matrix is defined to be $\hat X=T_1W_1^\top$. Using all these equations we get that the error matrix $E=X-\hat X$ is
$$
E = X - \hat X = X - T_1 W_1^\top = T_2 W_2^\top = XW_2 W_2^\top
$$
The rows of $E$ are the error vectors, that is the difference between the original data vectors—the rows of $X$—and the reconstructed vectors—the rows of $\hat X$. Therefore the trace of $EE^\top$, which is the sum of the squared lengths of the error vectors, is the total squared reconstruction error. 
So what is the trace of $EE^\top$?
We know that
$$
E^\top E = W_2 W_2^\top X^\top XW_2 W_2^\top
$$
and using $(N-1)S=X^\top X$, where $S$ is the covariance matrix of $X$, in the above equation
$$
\begin{align}
E^\top E &= (N-1)W_2 W_2^\top SW_2 W_2^\top \\
         &= (N-1)W_2 W_2^\top W_2 D_2 W_2^\top \\
         &= (N-1)W_2 D_2 W_2^\top
\end{align}
$$
where $D_2$ is the diagonal matrix with the lowest $(M-L)$ eigenvalues of $S$. Finally, using the properties of the trace we get
$$
\begin{align}
\text{trace}(EE^\top) &= \text{trace}(E^\top E) \\
                      &= \text{trace}((N-1)W_2 D_2 W_2^\top) \\
                      &= \text{trace}((N-1)W_2^\top W_2 D_2) \\
                      &= \text{trace}((N-1) D_2)
\end{align}
$$
This is telling us that total squared reconstruction error is just the sum of the lowest $(M-L)$ eigenvalues of $S$ multiplied by the constant $(N-1)$. Therefore minimising the total squared reconstruction error is equivalent to maximising the sum of the $L$ highest eigenvalues of $S$, which in turn is equivalent to maximising the variance of the first $L$ principal component scores.
