Suppose I have a set of predictors, I know some of them maybe useless, but I am not sure.

So, I build multiple versions of predictor set, that each version contain/not contain some of the predictors that I am not sure about. Then, for each version, I use cross-validation (CV) to tune a same learning model. I then calculate the CV error.

Can I use the predictor set with smallest CV error? Does it introduce bias or overfitting? Why?

Will it help, if a separate test set is used?