I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a split between those who perform SMOTE before or after PCA. Yes, i am very confused, but even authors of published papers seem to be confused as well about this (Me being a newbie). Even here (on CrossValidate) there are different answers for the same Q about SMOTE and PCA, and the gist is, see what suits your data. Anyway, I know how to perform each by it's own, but don't know how to do PCA in each fold, for example in Kfold, or any other cross validation method.

1- Is it possible for someone to share with me how to do PCA inside Kfold? Through a loop or pipeline maybe?

2- I personally have a bipolar opinion on the SMOTE/PCA thing: a- I think that PCA should be performed before SMOTE so that the dimentionality reduction is done on the main (real) training data before we sample it (In my case upsampling). b- My other thought on it, is that if you upsample and then perform SMOTE then your dimentionality reduction is based on "non-biased" group sample (where both groups now have same number). My Q here is, if i CV (cross validated) PCA, can i include SMOTE in the mix?

3- Last Q, I saw a video of a prof. who said that if you test/split your data and then CV PCA on the training set, this is wrong. Since cross validate e.g. Kfold already does the train/test split for you. Is this the case? If so, then how would i eventually select my X_test X_train y_test y_train for further analysis?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.