When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use feature selection and when to use dimensionality reduction?
Some mean by dimensionality reduction what is usually called data reduction. If this applies to you, then the main choices are
- Do you want to use unsupervised learning to reduce the number of parameters that need to be estimated so that you can avoid overfitting? If so then data reduction is for you.
- Do you believe that parsimony is appropriate and you want to use $Y$ to obtain a reduced model, risking more overfitting and model uncertainty because of phantom degrees of freedom? Then use traditional feature selection. This does have the advantage that results are easier to interpret, but this is somewhat because phantom degrees of freedom are being ignored.