0
$\begingroup$

Based on my limited understanding, there are in general two reasons why feature selection is used:

  1. Reducing the number of features, to reduce overfitting and improve the generalisation of models.
  2. To gain a better understanding of the features and their relationship to the response variables.

In most of the articles about feature selection focus is on reducing noise and to find a balance between bias and variance.

My question is - Are there any cases where higher noise(not so important features) in model will actually be helpful?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This question is rather broad and expecting an answer to this will only be based on personal opinion. Suggest you focus on a specific problem and explain why noise would be useful in that scenario. $\endgroup$
    – Arun Jose
    Commented Aug 17, 2016 at 6:23
  • $\begingroup$ Yes. Owen Zhang actually suggests adding noise to some features in order to prevent over-fitting. This is in this video youtube.com/watch?v=LgLcfZjNF44 at around 39:55. $\endgroup$
    – josh
    Commented Aug 17, 2016 at 12:44

1 Answer 1

Reset to default
0
$\begingroup$

It seems to be subjective question and answer will change from case to case. There could be scenarios, where just to gain extra predictive power, we can include some weak predictors(I hope you are referring to them, when you say "not so important features"). But most of the time, we would avoid that, as it causes overfitting and performance might fluctuate while validating on the test dataset. If you can help with your case, the community can give more precise answer.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.