I'm currently looking through Chapter 6 of An Introduction to Statistical Leanring by Gareth James. I am working through the Chapter 6 lab regarding Principal Component Analysis.
In the lab we first use k-fold cross-validation with 10 folds in order to find the optimal number of principal components to use. There are 19 components in total, and cross-validation shows that the lowest average MSE across all folds is achieved for 18 components, although 5 ish seems to capture most of the variance.
This all makes sense to me, but the next stage of the lab is slightly confusing. The book then goes on to say "We now perform PCR on the training data and evaluate its test set performance."
The data is now split into a single training and test set (split 50% between the two), and the 10 fold cross-validation process is repeated, this time revealing that 6 principal components gives the lowest training MSE. The lab then calculated the test MSE for a linear regression model containing these 6 principal components.
The final stage doesn't make sense to me. Why, once performing the initial cross-validation do we repeat the process on only half the data-set? Is it because, although a training-test split was used in the first cross-validation process, there was some overlap between the training and test splits. Is the purpose of the repetition of the cross-validation to be able to calculate a test MSE which is completely independent of the training data?
And as a final question, is the reason the lowest training MSEs for the first and second cross-validations different because less of the data was used in the second instance?