Preprocessing training data with autoencoders before training a ML model

Question

I have a 300D training data set and I want to use autoencoders to reduce the dimensionality before running a machine learning model on this data set.

In the classical dimensionality reduction technique PCA it is recommended that PCA is run only on the training set, and testing/validation sets are "predicted" with it.

How is the procedure for autoencoders? For example, I want to reduce the dimension of my training data from 300D to 10D, which means 10 neurons in the hidden layer of a neural network (autoencoders). My question is, should I run autoencoders on the training/validation/testing sets separately?

Answer 1

You should not ‘run’ your autoencoder on them separately. Fit it to your training set, then apply the learned transformation to all three sets.

This is exactly the same as what you should do with PCA. In fact, PCA is related to a simple linear autoencoder. But the advice is more general. Always fit everything to your training set.