0
$\begingroup$

How important are hyper-parameters in svm based RFE feature selection?

For feature selection using RFE (recursive feature elimination / selection), I have seen some publications where only "external" aka "outer" cross-validation is performed to estimate performance. The inner-cross-validation step is skipped and static hyper-parameter values are used instead, such as c=1 and gamma=0.1.

In other cases, gamma is set by heuristic such as 1/num_features (however, if using individual predictors, then num_features is always 1, so g=1 anyway). Also, in such cases, usually the features are all scaled 0 to 1.

Is this methodology a sound and scientific cross-validation method for feature selection? What kind of feature selection issues could be caused by not performing a parameter sweep with an algorithm such as grid search and instead using static parameters to estimate general performance to select the best features?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.