What is the name of this algorithm for choosing which features to include in a neural network? Once I read about one kind of neural network (for classification or feature selection) for a supervised training where you start with all inputs, then you proceed with a training step and randomly (or via criteria) you remove one input feature and then proceed with the training step. If this helps to improve the model, you remove this feature but if this is not the case, you reclaim this feature and remove others and evaluate again.
Also, you start with 1 feature and proceed with the train step. as the process before, you add new features and evaluate if this helps to model or not.
Do you remember the name of this kind of algorithm and have some article describing it or using it?
I have made my own research, and search on my browsing history with no success. Some kind of  "pull back" or "push forward" but not back-propagation.
 A: It's called stepwise selection and is generally not recommended. Moreover, with computationally intensive models like neural networks it is rather inefficient strategy. The standard, and giving much better effects, way of doing it is using some kind of regularization. You could use dropout, $L_1$ or $L_2$ regularization, or many other approaches.
A: I guess you are referring to a family of feature selection methods called wrapped methods.
In particular, what you call "pull back" or "push forward" are respectively Backward Elimination and Forward Selection.
I suggest you also to have a look at Recursive Feature elimination.
In the following link you find a quite clear description of them plus a comparison with pother feature selection techniques, so that you can better choose what suits you better.
https://www.analyticsvidhya.com/blog/2016/12/introduction-to-feature-selection-methods-with-an-example-or-how-to-select-the-right-variables/
example code are in R.
In case you prefer python here is it another valid link:
https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/
Hope this helps
