I have m examples and d features where m<<d. So I managed to compute the eigen value and corresponding its eigen vector ... I want to compute the reconstruction error for various value of principal components , lets say n principal componets (n eigen vector corresponsing to first max n eigen values)

So My original dataset has shape

(m, d)

If I take n principle components where n < d, the transformed matrix matrix would have shape

(m, n)

So Since the shapes mismatch I really wasnt able to find any way to compute the reconstruction error


1 Answer 1


Like you said, you cannot compute the error in that fashion because the dimensional space is not the same, you would need first to 'reconstruct' your new-reduced space to the old-bigger dimensional space and there you can compute the error as you said.

For the dimensionality reduction we do:

$T = XP$

for the reconstruction we do:

$\hat{X} = TP^{'}$

where $X$ is the original data, $T$ is the reduced space (formally called scores), $P$ are the loadings (PCs) and $\hat{X}$ is the reconstructed space.

Finally, we commonly express the error as:

$Error = (X - \hat{X})^{2}$


How do we interpret Error:

Before, first we denoted $X$ as our matrix of original data $\mathbb{R}^{nxk}$ ($n$ data points with $k$ dimensions each), and then we moved to the reduced space $\hat{X} \in \mathbb{R}^{nxp}: p<k$

When we compute the error, we are computing the distance (norm) between the original ($p$ dimensional) and the proyected ($k$ dimensional) observation.

  • $\begingroup$ $X$ and $\hat{X}$ are both matrices. What operation do you mean when you write $(X-\hat{X})^2$? Perhaps you're referring to some type of matrix norm? $\endgroup$
    – Sycorax
    Feb 19, 2021 at 2:02
  • $\begingroup$ Yes!, I edited my answer $\endgroup$
    – Ralphns
    Feb 24, 2021 at 15:13
  • $\begingroup$ Specifically you want either the Frobenius norm $||\cdot||_F$ or (less likely but still correct) the spectral norm $||\cdot||_2$. Either will do here; see people.tamu.edu/~sji/classes/PCA.pdf. $\endgroup$ Apr 2, 2021 at 3:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.