When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use feature selection and when to use dimensionality reduction?


Some mean by dimensionality reduction what is usually called data reduction. If this applies to you, then the main choices are

  1. Do you want to use unsupervised learning to reduce the number of parameters that need to be estimated so that you can avoid overfitting? If so then data reduction is for you.
  2. Do you believe that parsimony is appropriate and you want to use $Y$ to obtain a reduced model, risking more overfitting and model uncertainty because of phantom degrees of freedom? Then use traditional feature selection. This does have the advantage that results are easier to interpret, but this is somewhat because phantom degrees of freedom are being ignored.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.