3
$\begingroup$

I'm trying to apply logistic regression to the data with binary predictor. But some of my variables are numerical and some are categorical. If I just do this in R I get the model where for every categorical variable I have coefficients and p-values for for variable's possible values except for the first.

How can I interpret such model? And what is the best way of finding best model for such problem?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Your coefficients (which are log odds-ratios) are for a particular level of each variable stated relative to a reference category. So there is only one coefficient for a two-level predictor, as the coefficient represents the difference in the log-odds between two groups.

That is, if you are interested in modelling likelihood of having a disease (outcome) based on gender (Male/Female) and smoking status (smoker/non-smoker), then the coefficient for gender is the log odds ratio for e.g. Males relative to Females; and the coefficient for smoking is for smokers relative to non-smokers.

For factor variables, R chooses reference groups by default as the first level of that category, which does depend on the order of the levels. (When importing factors, R does this alphabetically, which is why female and non-smoker are the reference categories above. But sometimes the levels may have been applied in a different way, so it's important to check.) See relevel if you want to see how change reference categories.

Improve this answer
$\endgroup$
4
  • $\begingroup$ But how can I use these results then? I mean, for numeric variables I can use these coefficients to construct rule for classification.I have a number of both categorical and numerical variables. And I need to find variables and rule which provide best AUC as well as to calculate their odds ratios. And also I can use p-values for variables. But here for categorical variable with, for example, 4 possible values I have 3 p-values.. $\endgroup$
    – Oleg
    Commented Mar 23, 2013 at 0:33
  • $\begingroup$ @Oleg -- these are some quite big questions, and you may need to track down a textbook for details, or pose these as a new question. I'll give a few directions to some of the pertinent points for this particular question in the following comment(s). $\endgroup$
    – James Stanley
    Commented Mar 23, 2013 at 5:58
  • $\begingroup$ (1) Coefficients from this model are the log odds ratios (which is what the model fits) -- you can exponentiate them to get the odds ratios. (2) To get a hypothesis test for whether a multi-level factor (eg a four level factor with three coefficients) is associated with an outcome (at all -- a test that all of whether all of the beta coefficients might be zero in the population) one can use Type 3 tests (Wald or likelihood ratio tests would be one option for considering the impact of a set of coefficients.) $\endgroup$
    – James Stanley
    Commented Mar 23, 2013 at 6:04
  • $\begingroup$ BTW, I think the question parts "How can I interprete such model?" and "And what is the best way of finding best model for such problem?" are pretty comprehensive, so I can't really attempt an answer. $\endgroup$
    – James Stanley
    Commented Mar 23, 2013 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.