3
$\begingroup$

Once I read about one kind of neural network (for classification or feature selection) for a supervised training where you start with all inputs, then you proceed with a training step and randomly (or via criteria) you remove one input feature and then proceed with the training step. If this helps to improve the model, you remove this feature but if this is not the case, you reclaim this feature and remove others and evaluate again.

Also, you start with 1 feature and proceed with the train step. as the process before, you add new features and evaluate if this helps to model or not.

Do you remember the name of this kind of algorithm and have some article describing it or using it?

I have made my own research, and search on my browsing history with no success. Some kind of "pull back" or "push forward" but not back-propagation.

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ This is stepwise regression and tends not to be recommended by statisticians. $\endgroup$
    – Dave
    Sep 29, 2021 at 0:03
  • $\begingroup$ @Dave thanks, its not for development, only for comunicative reasons. Thanks for advice $\endgroup$
    – L F
    Sep 29, 2021 at 0:05
  • $\begingroup$ Pullbacks and pushforwards have nothing to do with this, these are terms from automatic differentiation. $\endgroup$
    – Firebug
    Sep 29, 2021 at 0:14
  • $\begingroup$ yeah I know @Firebug, I am mathematician but not native english speaker, so I was unable to find the correct word for "adding one input to next model/ deleting one input to next model". thats why I said "but not back-propagation" meaning that those terms do not reffer to the update weights process due b-propagation. $\endgroup$
    – L F
    Sep 29, 2021 at 0:17
  • 1
    $\begingroup$ What you want is probably related to Recursive Feature Elimination (RFE). Look it up, if it's not RFE itself it will be listed next to it in machine learning libraries $\endgroup$
    – Firebug
    Sep 29, 2021 at 0:28

2 Answers 2

Reset to default
1
$\begingroup$

I guess you are referring to a family of feature selection methods called wrapped methods.

In particular, what you call "pull back" or "push forward" are respectively Backward Elimination and Forward Selection. I suggest you also to have a look at Recursive Feature elimination. In the following link you find a quite clear description of them plus a comparison with pother feature selection techniques, so that you can better choose what suits you better.

https://www.analyticsvidhya.com/blog/2016/12/introduction-to-feature-selection-methods-with-an-example-or-how-to-select-the-right-variables/

example code are in R.

In case you prefer python here is it another valid link:

https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/

Hope this helps

Improve this answer
$\endgroup$
2
$\begingroup$

It's called stepwise selection and is generally not recommended. Moreover, with computationally intensive models like neural networks it is rather inefficient strategy. The standard, and giving much better effects, way of doing it is using some kind of regularization. You could use dropout, $L_1$ or $L_2$ regularization, or many other approaches.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.