How to choose reference category of predictors in logistic regression? [duplicate]

Question

I am struggling to decide which reference category I should define in my logistic regression model. When I define "mandatory school" as a reference in the variable education the results seem different compared to when I define "High school" (the significance disappears).

Anova test (likelihood ratio test) of models with and without education was significant.

I am aware that similar posts exist but I did not find a sufficient answer. My aim is to identify if education has an effect on the outcome. As education is an ordinal variable it is naturally ordered from lowest to highest education - but which education level should be the reference?

I know that there are some strategies, e.g., using the category that is most frequent in the data. However, as my significance values for the different education levels vary depending on which reference level, I do not just want to use the level that brings the most significant levels ...

Answer 1

A few things.

First, the significance does not disappear. You are asking different questions, so you get different answers. Comparing high school to to the other levels is three questions. Comparing mandatory school to the others is also three questions, but different ones. The overall importance of education is the same either way.

Second with ordinal variables, I often like to prefix the words with numbers for the order. So: "1:Mandatory school", "2:High school" and so on. The output is given in alphabetical order, and this makes it easier to see.

Third, with ordinal variables, one possibility is to pick the most common category as the reference. Another is to pick either the lowest or highest level (here, either "mandatory school" or "PhD". Another is to use optimal scaling -- this is not so commonly used, but has some nice properties.

Fourth, I notice you also have age and that a) It is categorized and b) the lowest category is < 18. For a) it would be better to use age in years, if you have it. For b) this is a problem because it will be impossible for people who are under 18 to have PhDs, and very hard for them to have BAs.

Finally, do see Noah's answer, which addresses your question most directly.

Answer 2

My aim is to identify if education has an effect on the outcome.

If that is the case, then you should do a likelihood ratio test between a model with education and a model without it. That is agnostic to coding and will give the same p-value regardless of coding. It also answers your research question directly with a single test.

To to characterize what the effect of education on your outcome is, you should use a post-estimation procedure like computing and comparing average adjusted predictions (e.g., using marginaleffects) rather than trying to engineer your model to provide interpretable coefficients. Such approaches are agnostic to how the variable is coded and which is the reference category.

Choosing the reference category is only important if you want to interpret the model coefficients, in which case it makes to choose the reference category based on the research question you want to answer. In your case, it seems that your research question does not have to involve regression coefficients and you can and should use methods that are agnostic to coding. See also my answer here on a related topic.

Answer 3

I would typically omit the most common or default category for education. The remaining coefficients should be interpreted relative to that category, which serves as the baseline.

It’s not surprising or problematic that different comparisons have statistical significances that vary. Compared to only HS (model 1), folks with higher levels are increasingly and SS more likely to pass, as expected (assuming MS corresponds to more schooling than HS but less than the other two). Compared to MS (model 2), the difference with college is not distinguishable, though the others still are SS from zero.

But ultimately, the choice depends on the comparison you want to make. There are several options for answering the question if higher pass rates are associated with more education. I would test the joint null that all three coefficients in model 1 are greater than zero. But there is also a dose-response pattern that may be interesting to explore, which would entail ordering the coefficients as well: $0 < \beta_{MS} < \beta_{College} < \beta_{PhD}$. There are certainly others that might make sense as well.