1
$\begingroup$

I am trying to understand the impact of feature selection on neural networks.

If I have n features and n' number of features are redundant, and I eliminate those n'.

Do the remaining number of features I use for deep learning have a positive/negative effect on accuracy or does feature selection here simply reduces time to train the model since there are lesser number of parameters now?

$\endgroup$
3
  • $\begingroup$ How redundant is “redundant”? Do you mean the same measurement in feet and meters, the same measurement in feet twice, or just high correlation but not perfectly redundant? $\endgroup$
    – Dave
    Jun 3 at 16:45
  • $\begingroup$ @Dave the features have a variance of zero and their feature importance after running Random Forest also comes out to be zero. $\endgroup$
    – jpj
    Jun 4 at 11:45
  • $\begingroup$ If the features are constant, they do not contribute to distinguishing between the classes. $\endgroup$
    – Dave
    Jun 4 at 12:32
1
$\begingroup$

It depends.

Most likely the network is probably going to pickup some of the noise from the 'redundant' variables so you could see an increase in validation forecast accuracy and a decrease fit accuracy if you drop them. You could also see nothing much happen if the variables are truly redundant and the network isn't really doing much with them anyway.

So, it depends.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.