I'm using sklearn
's linearRegression
model. After regression is complete, I get back a set of features and a set of coefficients. Referring to this post, I found how to map each feature to its corresponding coefficient. If I understand correctly, these coefficients are the coefficient of variables in the equation that represents the independent variable.
So, for instance:
If I am modeling the selling price of a toy based on time since launch (how many days ago was this toy released in the market) and weight of toy
and
i get -0.3 for time_since_launch and +200 for weight,
then weight is the dominating factor.
If this understanding is correct, I am confused about the results that my model is giving me.
My methodology:
I am performing stepwise linear regression (I know it has flaws, but I expect it to still give me reasonable results). Each iteration of the stepwise process iterates through each of the 150 features, performs regression with the {current set of features+ current loop feature} and then includes the feature that gave the lowest loss.
Now, the stepwise iteration is spitting out features in an expected order, i.e., features that I expected to have high impact were being added to the model first. However, when I mapped the coefficients and feature names, something seemed very off. Some of the features that were added to the model early on, had very low coefficients (and some features that were added later, had high coefficients).
Can anyone spot the problem?