1
$\begingroup$

Suppose I have a logistic regression output as follows.

Coefficients:
                 Estimate   Std. Error z value  Pr(>|z|)
(Intercept)     -5.8700887  0.0506436 -115.91   <2e-16 ***
f1               0.0511820  0.0005718   89.51   <2e-16 ***
f2              10.8470102  0.0992892  109.25   <2e-16 ***
f3               0.3762872  0.0108390   34.72   <2e-16 ***

In this case, which variable play the most important role in the model? Is it f1 (the lowest coefficient) or f2 (the highest one)?

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
1
$\begingroup$

There is no simple answer to that, as long as you don't have normalized predictors. If one of the predictors is a length, then the estimated coefficient will change dramatically, depending on whether you entered the length's in centimeters, inches, meters, kilometers or astronomical units. At the same time, the "importance" of the predictor does not chance depending on units.

So you might check if you want to normalize your predictors so that they become more comparable. After that, the size of the coefficient is the number of standard deviations that go into the model and that can be regarded as one possible definition of "important role".

Improve this answer
$\endgroup$
2
  • $\begingroup$ Suppose I normalized the input (f1,f2,f3) to unit vectors already. In this case, does it mean that "f2" is the most important one, i.e. if changing each variable just 1 unit, the change on f2 will have the highest impact to the predicting value? $\endgroup$
    – mommomonthewind
    Commented Mar 12, 2018 at 13:52
  • $\begingroup$ Yes, a one standard deviation change in f2 will have an impact of 10 points on the logistic scale while f1and f3 will have less then 1 point in the logistic scale. Do not expect coefficients in the neighborhood of 10 in standardized predictors in real life usage, though. $\endgroup$
    – Bernhard
    Commented Mar 12, 2018 at 15:11
1
$\begingroup$

You might also consider dominance analysis or relative weights based approaches.

These approaches do not require a manual rescale but also do not offer quite the same information as the approach above. They comment on the percentage of the McFadden (or Estrella) pseudo-R2 is explained by each predictor.

This can make the results easier to compare across predictors but, again, provides different information (no longer comments on changes in log-odds over the predictor; now refers to amount of information in the binary dependent variable is attributed to each predictor).

Also a handy online calculator for the relative weights method is available.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.