Partitioning training data for dimension reduction and classification

Question

Let's say I want to test the performance of my dimension reduction + classification pipeline. To do this, I will use k-fold cross validation. I know that performing dimension reduction on the complete dataset before creating the k folds is bad due to overfitting. To avoid this, the k folds for training and testing are created first. My question is the following: how should my dimension reduction + classification pipeline learn? I see two options:

1. Take my training data, divide it in two (how many samples go to each is to be determined). Use one subset to learn the dimension reduction mapping. Then, pass the other set through the learned mapping and use the reduced features to learn the classifier. Now, none of the steps have overfitted.
2. Take my training data, apply my dimension reduction to it i.e use the same data for learning and reducing. Use the reduced data to learn the classifier.

I tend to prefer approach (1) given that no overfitting occurs. With method (2), I would run into issues when I want to use my dimension reduction + classifier pipeline on new data.

Is approach (1) the correct one? Is there another way to do this? I'm not making assumptions on whether the dimension reduction is supervised or not.

Answer 1

If you use approach 1, you'll need to reserve a third independent set for validation/testing of the ready-to-use model. Other than that, it is perfectly finde.

People usually try to avoid that additional split (you'd need yet another one if you do data-driven hyperparameter optimiziation!) and therefore go for approach 2 which leaves the second split for validation/testing.