Can PCA be reversed to predict original variables based off a random point in PCA space?

Question

So, I recently read this answer: https://stats.stackexchange.com/questions/34724/reversing-pca-back-to-the-original-variables

I wanted to make sure that I'm understanding this correctly. So, normally I fit PCA to a dataset, like the digits dataset. Then, I plot the resulting projections (that were created for my original dataset) onto a 2D or xy plot. My understanding was: I can select a random point x,y in the plot (that does NOT exist in the projections data I plotted, or is not a plotted point), and then reverse this x,y point with PCA to "PREDICT" the original data (e.g. predict what digit that point represents).

Is my understanding of this correct? If so, doesn't this basically make PCA a prediction algorithm?

Also, does this understanding also apply to the following UMAP article: https://umap-learn.readthedocs.io/en/latest/inverse_transform.html ?

Answer 1

Your understanding is only half correct. PCA (Principal Component Analysis) is a dimensionality reduction technique that finds underlying patterns in data to reduce the number of features in a dataset. It does not make predictions; rather, it generates new dimensions from existing data for further analysis or visualization.

PCA can be reversed (or "back-transformed") to reconstruct the original data, but this does not imply predicting a new point in the PCA space. To do so, you would need to use a supervised learning algorithm, such as a regression model trained on the PCA-transformed data, predict a new point (in the PC space), and then use the inverse transform to project the predicted principal components to the original space.

I'm not as familiar with UMAP, but it appears to be similar in that it can be used to reduce the dimensionality of a dataset and then inverted to reconstruct the original data. This is not, once again, the same as making a prediction.

I hope this helps!

Answer 2

You're missing a step: fitting some kind of predictive model. Once you transform your PCA feature space into the original features (which need not be unique unless you use all of the PCs), you have to have some way of interpreting that.

For the MNIST digits to which you allude, that could be a convolutional neural network, through which you pass the $28\times 28$ picture you derive from your point in the PCA space. For other data, you might prefer another model.

Ditto for UMAP.