There's many questions on related topics but I have been unable to find one that precisely answers my question.
Let's say I'm performing a regression on multiple predictor variables $x_1...x_n$ for explanatory purposes. My intent is that the size of the coefficient determines the importance of that variable in the outcome $y$. I want to be able to say something like "This coefficient $b_2$ is the largest, therefore this variable $x_2$ has the biggest effect on $y$". I do not care about the intercept.
My $x$ variables are of different types: Categorical binary (either 0 or 1), continuous on [0,1] and continuous over $(-\infty,\infty)$ and I am not sure how much normalisation or standardisation is necessary to make the coefficients of these variables comparable to each other in the model.
Below is a list of statements in decreasing order of how sure I am they are true.
It is valid to compare the coefficients of:
- The categorical variables with each other
- The continuous variables on [0,1] with each other
- The continuous variables on [0,1] with the categorical variables
- The continuous variables on $(-\infty,\infty)$ with each other, providing they are standardised.
- The standardised continuous variables on $(-\infty,\infty)$ with all the other variables
My questions are:
1. Are the above statements true?, and
2. If they are not true, what transformation is necessary (if any exists) to make them true?
multiple regression importance
. One popular easy measure is the relative size of the squared part correlation of the predictor with the Y. For each predictori
in the model, compute importanceI_i = SSE_without_i - SSE_full_model
. Then normalize by dividing by the sum of the I's of all the predictors. $\endgroup$ – ttnphns Aug 28 '18 at 10:35