Backward feature selection with CV model selection I am thinking about doing the following to a data set with $N$ samples and $m$ features
1) Train using semi-supervised learning and cross validate on labeled data using LOO-CV to select the best model.
2) Once we have the best model, eliminate one feature and go again to 1. Search again for the best model. 
3) Stop when you have $n < m$  features and the best model
Do this tend to overfit?
Edit:
Would it be better if I adjust the model with all the features and then perform backward feature selection only?
 A: *

*if you optimize your model in some way, you need a validation step that is outside (= independent) of the optimization. Thus, if you optimize using CV results, a double or nested validation set-up is necessary. 

*While CV can help to reduce overfitting for a given step, the proposed iterative optimization will tend to overfit to your data set. 

*Instead of doing the backward feature selection with the overfitting problem, you could think about e.g. using LASSO regularization which will also produce a feature selection by shrinking coefficients to 0.

*Semi-supervised models assume that unlabeled and labeled data come from the same distribution. This is often violated [Berget, I. & Næs, T. Using unclassified observations for improving classifiers, J Chemom, 18, 103-111 (2004). DOI10.1002/cem.857]
You need to be more careful if you optimize your model: the semi-supervised approach in itself already increases the risk of overfitting as unlabeled data essentially cannot correct the model, it can only refine estimates of where data points are expected. 
A: CV results in a series of models and it's not clear why you would use CV to select the model.  I think a better approach is to decide how you want to develop the model, fit the model in the whole dataset, and use the optimism bootstrap to unbiasedly validate that model using resampling.
