Standardization and explanatory variables of different domains in Multiple Regression

Question

There's many questions on related topics but I have been unable to find one that precisely answers my question.

Let's say I'm performing a regression on multiple predictor variables $x_1...x_n$ for explanatory purposes. My intent is that the size of the coefficient determines the importance of that variable in the outcome $y$. I want to be able to say something like "This coefficient $b_2$ is the largest, therefore this variable $x_2$ has the biggest effect on $y$". I do not care about the intercept.

My $x$ variables are of different types: Categorical binary (either 0 or 1), continuous on [0,1] and continuous over $(-\infty,\infty)$ and I am not sure how much normalisation or standardisation is necessary to make the coefficients of these variables comparable to each other in the model.

Below is a list of statements in decreasing order of how sure I am they are true.

It is valid to compare the coefficients of:

• The categorical variables with each other
• The continuous variables on [0,1] with each other
• The continuous variables on [0,1] with the categorical variables
• The continuous variables on $(-\infty,\infty)$ with each other, providing they are standardised.
• The standardised continuous variables on $(-\infty,\infty)$ with all the other variables

My questions are:

1. Are the above statements true?, and

2. If they are not true, what transformation is necessary (if any exists) to make them true?

Answer 1

Standardizing continuous predictor by subtracting their mean and dividing by their standard deviation is a procedure that is often used to put all predictors on a common scale. In that sense, the magnitude of the estimated coefficients could be compared to see which predictors are associated with the largest change in your outcome when all of them are increased by the magnitude of one standard deviation.

However, there are some important issues associated when doing this standardization. Some of them have been highlighted above. An additional issue is that such standardization is data specific. I.e., if someone else did in another dataset, he would standardize the predictors differently because in his sample the mean and standard deviation of the predictor would be different than in your sample.

For more on this, check also this paper by Andrew Gelman: Scaling regression inputs by dividing by two standard deviations.

Answer 2

The reason why you're not finding answers to these questions is definitions: neither importance nor effect of a variable are defined statistical terms. Regression coefficients measure the effect of some unit of change in a variable, and scaling transformations simply change that unit. What people refer to as "importance" is mostly determined by the practical, moral and philosophical issues surrounding the problem, and not only by the regression coefficients.

For a practical example, let's say $y$ is some clinical outcome of interest (disease risk or continuous measure), and you ran a regression on potentially causal factors $x_i$. Here's some properties of $x$ that could be of interest to you, but will not be reflected in the $\beta$s:

1) Is a variable "important" if it cannot be changed? Height (in adults) has an effect on various disease risks, but short of eugenics we have no way to change it in well-nourished populations.

2) Is a variable "important" if it is not causal for $y$, yet strongly associated with it? For prediction purposes, it might, for explanatory - no.

3) Is a category "important" if it has a large effect (relative to reference), yet is very rare? Developing drugs against rare conditions could be very costly.

4) Is a variable "important" if we already know about its effects and have policies targeting it (e.g. smoking)?

...

Standardised coefficients and $R^2$ are useful because they provide a meaningful interpretation as "variance of $y$ explained by $x$". This partially takes care of issue 3), as rare categories or low-variation continuous features do not offer much predictive power. The drawback is that such measures depend on the distribution of predictors, and thus will not be comparable across populations or experimental settings.
As for the other issues, one can convert anything into a single "importance" objective, but that will depend on the particular user needs (and morals, virtues and everything else), so don't expect any universal statistical answer.