Are t-SNE and UMAP for dimensionality reduction compatible with cross-validation in machine learning applications?

Question

t-SNE:
Based on how I understand the original t-SNE algorithm, it requires a whole dataset for doing the transformation. That is, there are no distinct "fitting" and "transformation" steps like in PCA.

E.g., in PCA you can obtain transformation matrices from the training set and apply those to transform the test set or any new/unseen data later on. When I understand correctly, this is not possible with t-SNE. I.e., if you train a machine learning algorithm with t-SNE transformed training data and you want to then make predictions on a new data point (or test folds during cross-validation) you have a problem.

This is my understanding of the limitation of t-SNE. It's more of a dimensionality reduction technique for visualization, not necessarily building predictive models. My question is, is this true / still true? Are their t-SNE variants that behave differently so that they can be used in a prediction pipeline?

UMAP:

When I understand correctly, UMAP has a distinct training step, so it can be used in cross-validation and for building classifiers that you can use to make predictions on new data. Because here, you don't require the whole dataset for transforming new data points. Is this correct?

Answer 1

You seem to be getting it. PCA and UMAP give functions that can be applied to new data. In that sense, you can learn a PCA or UMAP transformation for several folds and then apply that transformation to new data.

Unfortunately, t-SNE does not work this way; t-SNE works on all data at once and does not produce a function to apply to new data. Thus, t-SNE is not a viable option for cross-validation where you apply a learned function to new data.