If we build a classifier based on a very small number of instances (say, fewer than 300) and the number of features we are using is very large (say, larger than 100k features). If we decide to introduce a feature selection step before building the classifier, is there a rule of thumb on how many features we should choose?

  • $\begingroup$ I would think this problem is too data specific for there to be any rule of thumb. $\endgroup$
    – Glen
    Mar 31, 2012 at 3:13

1 Answer 1


It depends on the type of your target variable. Frank Harrell gives some guidelines in his book, "Regression Modeling Strategies". You can find the discussion here (page 61): Regression Modeling Strategies


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.