I would like to make a PC/PLS regression and assess the resulting model's predictive power. The strategy is the classical splitting into training/validation/test sets, and using training/validation sets to fit the model and test set to assess the prediction performance.
My question is, can I use the whole data set to calculate the PCA if I then use only training & validation sets to fit the model parameters (i.e., regression weights and number of PCA components)?
Performing PCA on the whole data set would mean peek at the test set and might be considered cheating. However, if the data set is small one might not get an accurate estimate of the statistical structure in the data unless the whole set is considered, in which case it could be seen as legitimate (?)
I couldn't find any discussion of this over the web, so any pointers would be appreciated.