I am using RFE with logistic regression. I will also be doing cross validation with RFE (RFECV in sklearn) to get the optimum number of features.

I am not sure whether to use RFECV on just train data set or complete dataset (including test data). Can anyone throw some light on the right thing to do.

Second part of my question is: since RFE will eliminate some features including dummies of categorical features would it be right to drop a level/category when creating dummies? I mean if say feature CAT has 5 categories A to E, when I create dummies let's say I drop A and create 4 dummy variables for B-E. And RFE finds D & E unimportant and thus eliminates them. Wouldn't the logistic reg coefficients for B & C be wrong? As per this SO answer dropping some categories and keeping rest would not be right as it would inflate coefficients and p values of B and C.

https://stats.stackexchange.com/questions/24298/can-i-ignore-coefficients-for-non-significant-levels-of-factors-in-a-linear-mode

Please share your knowledge.

Thanks.