1
$\begingroup$

Feature selection reduce size of a problem - reducing compute time and space required to run classification algorithms.

While feature selection does decrease complexity, it does NOT have to improve classification accuracy.

Why does feature selection decrease complexity, but it does NOT necessarily lead to improvement of classification accuracy? Can you provide papers for that?

$\endgroup$
1
  • 2
    $\begingroup$ feature selection necessarily involves throwing out information, so if not done carefully can lead to worse performance $\endgroup$
    – shimao
    Commented May 27, 2018 at 13:06

1 Answer 1

1
$\begingroup$

If $X_1$ and $X_2$ are independent standard normal random variables, and $Y$ is $1$ when $X_1 + X_2$ is positive and 0 otherwise, then a model with both $X_i$s can achieve perfect classification accuracy whereas a model with only one of the $X_i$s won't have enough information to do so.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.