I am running a logistic regression to study the association between a three-level factor and a binary outcome, after controlling for covariates. My sample is small to medium (~800). The likelihood ratio test of adding the factor is not significant which, to my understanding, indicates the model without the factor fits the data as well as the model with the added factor, and therefore the factor can be dropped. Yet, the odds ratio of at least one level of the factor is significant (the profile CI does not include 1). I have seen people do the following and am wondering which is on better statistical footing:
- Report both the nonsignificant LRT and interpret the odds ratios of the factor levels. I have seen people argue that LRT may not be significant due to power and sample size and/or argue that the odds ratio should still be interpreted as they test differences between levels even if the overall factor is not significant. Some folks also interpret the Odds Ratios but then hedge (report everything but then throw in a line that it could be an artifact).
- Report only the nonsignificant LRT and do not interpret the significant odds ratios for the factor levels, under the rationale that they do not explain the data better and significant findings regarding levels of that factor could be artifacts.
I am leaning toward the second, yet would appreciate advice and sources which discuss whether to interpret significant findings for factor levels when the LRT is not significant. I see this as similar to the debate on this post whether to interpret comparisons or contrasts if an omnibus test is not significant. I see other posts discussing the discrepancy, just not the best way to interpret it.
EDIT: For others in the same situation (i.e., the LRT between logistic models after adding a factor is not significant, yet the odds ratio for a level of the factor doesn't include 1 and therefore looks significant)), based on the advice below and after reading other papers, I ran the factor level comparisons using Tukey post-hoc test with Holm-adjusted p values to adjust for running multiple comparisons (instead of just looking at the profile CIs). In all the cases were the LRT was not significant, the adjusted p values for factor levels were also not significant. So, this method (LRT between models along with post-hoc tests with adjusted p values) appears more consistent and makes it easier to conclude that any difference, implied by the odds ratio CI, between the factor level and the reference group is likely either small or an artefact. For others wondering how to conduct these post-hoc test for a logistic regression, you can use:
library(multicomp)
summary(glht(model, mcp(factor_of_interest="Tukey")), test=adjusted(type="holm"))
See here for more info on how to conduct and write this up.