I'm working on a project and I would like to know if the following strategy is good/correct. Sorry if this is a basic/stupid idea (I'm new to this).
The input is a dataset with 2.500 features and 1.000 instances. I have to apply a feature selection on this set. In the beginning I randomly make a learning set (which contains 70% of the original data) and a test set (with the remaining 30%).
Now I split the learning set in 10 folds (to do cross-validation), so we've split the learning set into 10 (more or less equal) parts and now we take 9 parts out of 10 to do a feature ranking (using chi squared), then we take the best 10 features (highest ranking) to do a model evaluation (using SVM) on the remaining part of the 'splitted learning set' (and we do this 10 times, so each part will be used as a 'test set').
Doing this '10 folds' argument gives us 10 feature rankings (with 10 model evaluation/accuracies). Out of these 10 results we can take the best feature ranking (according to it's accuracy based on the model or to take the sum of the feature rankings) and then finally test it on the 30% test set from the beginning.
Please give your opinion whether this is good or really stupid...