# Reversing PCA back to the original variables [duplicate]

I have a set of data that has $n$ samples described by $m$ variables. I do a PCA to reduce it to just 2 dimensions so I can make a nice 2D plot of the data. I understand that the $x,y$ coordinates (i.e., the PCA scores) for the plot are calculated by basically summing the products of the original data (after centering) by the loadings for each variable, so:

$$\mathrm{PC}_1 = X_1L_1 + X_2L_2 + ... + X_mL_m.$$

My question is, if I pick an arbitrary point in the PCA space (i.e. a value for $\mathrm{PC}_1$ and $\mathrm{PC}_2$, or $x$ and $y$ in my plot), is there a convenient way to translate that back to a set of the original values (i.e., $X_1,X_2,\dots,X_m$)?

Note 100% reversal is obviously not expected (since I'm only using 2 PCs), so a decent approximation is fine.

• Read the first chapter of Michael Greenacre's Biplots in Practice. Then read the rest of it when you understand that much! It is in essence a demonstration of how to do the type of interpretation from a biplot that you are asking for. Commented Aug 20, 2012 at 18:03
• [cont.] We can close this one as a duplicate of mine, because the latter thread is more comprehensive. What do you think? Or perhaps I should've posted my answer here; maybe it can still be moved somehow. I did not want to vote to close before discussing it with you, as your thread has a respected four-years-long history. Cheers. Commented Aug 12, 2016 at 22:36
• @amoeba: I'm completely agnostic about it. Whatever you think would be most helpful for the community. Commented Aug 13, 2016 at 1:33

Yes. Basically, what you did was to do: $$\mathrm{PC}=\mathbf{V}X,$$ where $\mathrm{PC}$ are the principal components, $X$ is your matrix with the data (centered, and with data points in columns) and $\mathbf{V}$ is the matrix with the loadings (the matrix with the eigenvectors of the sample covariance matrix of $X$). Therefore, you can do: $$\mathbf{V}^{-1}\cdot\mathrm{PC}=X,$$ but, because the matrix of loadings is orthonormal (they are eigenvectors!), then $\mathbf{V}^{-1}=\mathbf{V}^{T}$, so: $$\mathbf{V}^T\cdot\mathrm{PC}=X.$$ Note that this gives you exactly the same equation you cite for the recovery of the PCs, but now for the data, and you can retain as many PCS as you like.

• Ok, but does that apply for any arbitrary point? i.e. a point that wasn't in the original dataset? Commented Aug 20, 2012 at 19:25
• Yes, but it can only be used for interpolation. Suppose datapoint $x_{ab}$ is missing and that the data matrix is centered. The problem is to find $\mathbf{V}$. You can estimate the sample covariance matrix as: $$\hat{\Sigma}_{ij}=\alpha_{ij}\sum_{k}x_{ik} x_{jk},$$ where $$\alpha_{ij}=\left(N-1-\sum_{k}[\delta_{ab}(i,k)+\delta_{ab}(j,k)]\right)^{-1},$$ and $\delta_{ab}(i,k)$ is one when $i=a$ and $k=b$ and zero otherwise. Given this, the transformation matrix $\mathbf{V}$ is straightforward to obtain (you just have to estimate the SVD decomposition of this sample covariance matrix). Commented Aug 20, 2012 at 20:22
• Well, by theory $V^TV$ should be identity matrix, but in practice, it is quite off. The off-diagonal is ~0.0005 and some of the diagonal entries are as small as 0.5. May I ask why? Commented Apr 23, 2015 at 14:20
• That is strange. Typical algorithms have maximum errors on the order of ~1e-16 (see mathematica.stackexchange.com/questions/46283/…); maybe you should change the method by which you are obtaining the eigenvectors and eigenvalues? Commented Apr 26, 2015 at 18:39

I have a doubt about the above answers. Since after dimension reduction, we only know 2 principal components, and the rest principal components are abandoned. The projection matrix V is not a square matrix (not completely orthonormal, it is a semi-orthogonal matrix). Suppose n is the number of samples and m is the number of variables. $X$ is a $m$-by-$n$ matrix, $V$ is a 2-by-m matrix (whose rows are the top 2 eigenvectors of the covariance matrix of $X$), PC is a 2-by-n matrix. Then we have PC = VX. Then $VV^T$ is an identity matrix, but $V^TV$ is not. Thus $V^TPC=V^TVX$ cannot give us the exact original matrix $X$, since $V^TV$ is not an identity matrix.

• That's because there is no way to perfectly reconstruct $X$ from $V$ if you kept only $2<m$ principal components. One can, however, do a best possible reconstruction; it is given by $V^\top VX$. It's not equal to $X$. Commented Mar 23, 2016 at 20:34