I have an issue regarding cross-validation and dimensionality reduction, specifically in the realms of neuroimaging. What I found out so far is that: if I wanted to conduct ICA on a dataset in order to extract timeseries used for prediction- it should be done strictly in the cross-validation loop.
My question now is: would it be valid to take a representative part of my dataset (given my dataset is large enough) to conduct ICA, and then use these ICA-maps on the rest of the data to extract timeseries for prediction (no leakage between the dataset used for ICA and the rest of the dataset).
A little example: say I have 200 subjects addicted to cheesecake. Now I take 50 subjects from that population and conduct ICA. Then I take the ICA maps and use them for the rest of the 150 subjects, extracting timeseries and predict labels (cross-validated). If valid, this may be save computational cost. Besides, it is probably easier to implement (although, the sample size is shrunk)
I'm looking forward to your opinions on that!
