1
$\begingroup$

I have category (season, Time of day...) and numeric variables (temperature, humidity). I want to compare the coefficient to find out what variable has more impact on the dependent variable.

But the coefficients for the categorical variables are just against the reference category. I can't compare them with the numeric coefficients. How can I do that?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

You can only compare coefficients once you've standardize columns with variance, otherwise comparison is meaningless. So what you can do is one-hot-encode your categorical data and standardize your full matrix data (containing both categorical & numeric data) per column - divide each column by its variance, you can also shift by the mean but this is not compulsory. This scales every column, and then when you run your logistic regression, all the weights will acount for the same so you'll be able to compare them.

Hope it helps, Vince

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your answer. But I don't know How to standardize category variable by variance. Because the category （season, Time of day）have not size. It's just nominal. Can you give me an example? $\endgroup$
    – Pei Welkin
    Commented Jan 16, 2016 at 11:22
  • $\begingroup$ Let's say you have a column which is season: Winter (W), Spring (Sp), Summer (Su), Autumn (A). Most of the machine learning algorithms (e.g. logistic regression) work on vector features so you have to vectorize. First way: W=0, Sp=1, Su=2, A=2. This is not good because you induce a distance which is not native (why would A-W=2?), so most of the time you one-hot-encode -> you create a vector: W=[1,0,0,0], Sp=[0,1,0,0], Su=[0,0,1,0], A=[0,0,0,1]. Once you have this you can standardize those four columns as if they were numerical (compute variance and divide). $\endgroup$
    – Vince.Bdn
    Commented Jan 18, 2016 at 10:10
0
$\begingroup$

I personally find standardization difficult to wrap my brain around if I want to interpret the coefficients. What does a 1 standard deviation change in the binary spring variable actually correspond to? Could I even explain it my grandpa over a beer? I find these sorts of transformations more useful in settings where I am, or my audience is, less familiar with the baseline. For example, a one standard deviation increase in the number of murders conveys the seriousness of the situation very well.

I would advise you to search for the key words "logit" and "marginal effects" on this site. For example, the formulas can be found here. You will still have to decide on the right units to use for such comparisons for the continuous variables, but these formulas should help you proceed from there.

I am not sure what your field is, but imagine trying to describe effect of seasons and temperature on the probability that a person buys an ice cream using a logit model. I think it is still meaningful to compare the expected increase in the probability of a sale during summer (relative to winter baseline or even relative to spring) to the effect of 10 degree temperature rise, particularly if you model includes interactions between them. You can even compare the effect of a 10 degree change in summer versus 10 degree change in winter. You can even get fancier and look at how the effect of a 10 degree temperature rise on the probability varies with a marketing campaign in winter versus summer. You can even back out how much the temperature has to drop on Saturday to get the same reduction as the seasonal change from Saturday to Monday. I find these sorts of comparisons perfectly natural and interesting.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.