I have a dataset with a large number of features, which I want to reduce. Should I look for a method that identifies the most important ones and throw away the rest, or should I look for a method that attempts to create a smaller number of new features from the old ones?
Which method of dimensionality reduction is more common in practice: feature selection or feature extraction? Is one superior to the other? How should I know which one to prefer? Finally, is there any benefit in combining the two?