2
$\begingroup$

I would like to rank variables of a logistic regression model on the basis of their predictive importance.

The model has both categorical and continuous variables.

For this purpose, is it okay to assign say 1,2,3,4..... values to categories of a categorical variable and treat it as a continuous variable and then standardize it along with other continuous variables and get standardised estimates from logistic regression using the standardized variables as input to the model?

If the purpose is to find relative importance of variables of an already built model, is this approach alright?

$\endgroup$

2 Answers 2

2
$\begingroup$

While you can mess around with pseudo-R2s, I have never found them to be very informative or useful in a logit model. You also run into other problems when you compare logit models with different coefficients (I don't have an immediate reference but if you Google or look at CV for logit scaling factor you should get an idea).

Here are a couple of alternative approaches:

  • Estimate average marginal effects. You can use standardized continuous variables, but comparing continuous to categorical variables is inherently difficult because they are not on comparable scales. The only way I would feel comfortable making a categorical variable continuous is if you test the porportional odds assumption (that an increase from 1 to 2 is comparable to 2 to 3, etc.). Even then, you may gloss over information when you make it continuous. I would run the model both ways, and look at average marginal effects/predicted probabilities with the categorical variable set at the same levels for the categorical and continuous case, and see how the results differ. Also map it out to see how the predicted probabilities change based on different levels of the variables.

If you are unable to convince yourself and others that your categorical variables can be continuous, then you have a harder task. You could estimate predicted probabilities of the quantiles or deciles of the continuous variables, and compare them to the categorical variable.

  • Look at the predictive ability of your model. Look into the various metrics of specificity, sensitivity, area under the ROC curve, etc., and see how your prediction changes based on the different variables.

In the end, because there isn't a direct way to do this in a logit model, I would approach this more than one way and see if all the methods triangulate together. If they do, you're golden. If not the story is more nuanced and will take more thinking.

$\endgroup$
1
$\begingroup$

You could possibly do a seperate logistic model for each covar and check the percent of variation explained (R2). For logistic regression the procedure is a little bit different, and a simply R2 will not be sufficient, so you will have to look at pseudo-R2s like Nagelkerke's. By doing this you will see how much each of your covariates explains in your response variable and you can then rank them accordingly. I hope to see some other people chime in here, just one idea.

Note that when comparing pseudo-R2s you will want to give them a 95% CI through permutation tests like boostrapping. Google has lots more specifically for R, but for background.

And yes you can assign numbers to your categorical factors, but you must make sure they are coded as factors and not numericals. In R this is simple -- given dataset "dataset" and categorical data "data":

dataset$data <- as.factor(dataset$data)

Good luck!

$\endgroup$
11
  • $\begingroup$ Chris, after I assign numbers to categorical variable, I will standardize this variable along with other continuous variables and use them as inputs to my logistic regression model to get standardized estimates. Now I can rank all variables according to standardized estimates. Do you think this is a valid approach. Just confirming. $\endgroup$ Oct 10, 2014 at 13:38
  • 1
    $\begingroup$ @user45409, why would you like to standardize categorical variables? If you apply the same transformation to each number (1:4) they will stay categorical and be treated the same way. One of the reasons for standardizing variables is to treat values with no natural metric such that you can describe the increase of one SD. $\endgroup$
    – Chris C
    Oct 10, 2014 at 13:45
  • $\begingroup$ The logistic regression model coefficient will give the increase odds ratio for one unit increase in a variable. Whether or not this is 1:4, or the standardized estimates is irrelevant to the model. it will be the exact same. Hope this helps! $\endgroup$
    – Chris C
    Oct 10, 2014 at 13:46
  • $\begingroup$ I want to get standardized estimates so that I can rank all the variables according to their relative importance. That is why I wanted to standardize categorical variable. $\endgroup$ Oct 10, 2014 at 15:01
  • 1
    $\begingroup$ Don't standardize categorical data. That doesn't make sense. If you go with the R2 approach, it's better you don't treat a categorical variable as continuous - entering it in as dummy variables will make more sense. That is what @Chris is sugeesting. $\endgroup$ Feb 19, 2015 at 20:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.