2
$\begingroup$

If I have done a (binomial) logistic regression. Is it then possible from the coefficients to calculate a percentage of how much each of the variables affects the dependent variable.

Say that we have for example var1: 0.635, var2: 0.245, var3: 1.243. If we know that the depends variable Y in the data is 1 at 0.64 of the time. Can we then use this to calculated something a long the lines of:

Variable 1 has a 25% effect on Y, Variable 2 15%, and Variable 3 60%?

$\endgroup$

4 Answers 4

1
$\begingroup$

It seems like you are asking a basic question as to whether or not one can compute an analog to R square from linear regression to logistic regression or perhaps more generally to the generalized linear model. The basis for this being possible with linear models is that that the total sum of squares can be separated into two non-overlapping sums of squares (model sum of squares and error sum of squares). But for generalized linear models R square does not carry through as a measure of goodness of fit for a model. The analog is the deviance which can be used to measure the effect of variables on the model by looking at the change in deviance when the variable is added to the model.

$\endgroup$
1
  • $\begingroup$ I was thinking that since the coefficients indicate how much the odds ratio changes when the different variables are increased by 1 I should be able to use this to calculate some sort of percentage comparison between them. Say: If we have two variables we calculated that the probability of getting 1 increase from 0.64 to 0.70 if variable 1 is increased by one, and from 0.64 to 0.76 if variable 2 is increased. Then in a sense variable 2 has double the effect as variable 1 (0.12 compared to 0.06) and I'd like to say that it determines 66% of the probability and variable 1 33% of it... $\endgroup$ Commented May 29, 2012 at 11:30
1
$\begingroup$

You need to remember that the coefficients are dependent on the scale of the variable. Consider a logistic regression where x1 is height (of a person) in inches, what happens if you change this to height in meters or height in milimeters? The coefficient changes (and what a 1 unit increase means changes), but the overall effect of the variable will not change (unless there is major changes due to rounding).

I have had regressions where the coefficient for the most important variable was close to 0 (once it even printed out as 0.000 due to rounding), this is because the variable is something like the dose of a drug in milligrams and differences are in 100s or 1,000s of milligrams, so a one milligram increase is not very meaningful (for those I often recode the variable so that a 1 unit increase is more meaningful).

Sometimes people will standardize all the predictor variables so that a 1 unit increase is a 1 sd increase. This can make the coefficients more comparable, but sometimes the standardized variable becomes less meaningful.

You also need to consider relationship between the potential predictor variables.

$\endgroup$
0
$\begingroup$

Sounds like you are looking for something like relative risk or first differences. If so, calculate the predicted probability of observing a 1 with everything but your variable of interest set at its mean or median and your treatment at a low (or high) value, giving something like this

RR_rs = Pr(Y1 =r,Y2 =s|x1) / Pr(Y1 =r,Y2 =s|x)

Or first differences

FD_rs =Pr(Y1 =r,Y2 =s|x1)−Pr(Y1 =r,Y2 =s|x).

I suppose looking at some type of information criteria might partially get you what you want as well.

$\endgroup$
0
$\begingroup$

If you take the standard deviation of a coefficient into account (so that different types of scale are standardized), I would say a weight of importance of one variable is given by:

$$ \frac{|\beta_j|\cdot \sigma_j}{\sum_{i=1}^{m}|\beta_i|\cdot \sigma_i} $$

P.S.: Sorry, that I am 10 years late, with this suggestion

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.