I have been thinking about one thing after reading documentation from sklearn about Feature Selection for building prediction models (http://scikit-learn.org/stable/modules/feature_selection.html#rfe)
More specifically 1.13.3 --> Recursive Feature Elimination
The criterion used to eliminate variables is the
coef_, which is the value of the coefficients for the parameters of your model. If you loop this process, each time removing the last in the ranking that the method returns, you are supposed to be keeping the most relevant parameters for your model.
My question now is: what does the value of the coefficient have to do with its importance? It is just a number that is going to be multiplied by the actual value of the variable. So, variables like for example temperature in Kelvins can have values in some processes of 1200K (just as an example), while the pressure in the same process is maybe 5 bar. So it is quite easy that the coefficient that goes with temperature is going to be lower, so after being multiplied by 1200 it has a reasonable value to predict, let's say, some force in Newtons, but the pressure coefficient is going to have a smaller value. This does not mean the temperature is less relevant for the response than the pressure!
So, where is the fundament of the FSE then?