3
$\begingroup$

Context: I have a training dataset with 10000 features and i have selected the most important through a Random Forest. I used my subset dataset to train a Neuronal Net.

Problem: When i use the validation dataset should i just drop the variables from the RF subset dataset? Doesn't imply that the data came from the same distribution, so no need for a non-parametric approach?

$\endgroup$
2
$\begingroup$

The simplest thing to do is to drop them. You used RF for feature selection. You find the features that have the highest variable importance and then you carry on to use these features/variables to perform your task.

Using RF to do feature selection shows which variables are best/most important to solve the task at hand, the other variables are redundant and may cause overfitting.

This is also a dimensionality reduction, which may help with interpreting the results from the model that you are training, although it is usually difficult to interpret stuff from neural networks.

$\endgroup$
  • $\begingroup$ hello good sir :) $\endgroup$ – Soren Havelund Welling Jul 29 '15 at 22:40
  • $\begingroup$ Hej! Also remember to check out forestFloor! $\endgroup$ – Gumeo Jul 30 '15 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.