I ran a univariate logistic regression analysis on some risk factors for a dichotomous outcome. One of the variables is highly significant, but only has 5 outcomes in one of the groups. The confidence interval ends up being very wide when I run a multivariable regression. Is it appropriate to include it, or should I remove it from the multivariable model?
5
-
$\begingroup$ "only has 5 outcomes in one of the groups." which groups? $\endgroup$– Sextus EmpiricusJul 2, 2018 at 18:22
-
$\begingroup$ "The confidence interval ends up being very wide when I run a multivariable regression" is this where you obtained the high significance? If not, what model did you do that got the high significance? $\endgroup$– Sextus EmpiricusJul 2, 2018 at 18:24
-
$\begingroup$ Might be better to explain with more details on the variables: outcome is death (0/1). The predictor is disease x (0/1). So only 5 patients with disease x were alive, but over 20 were dead. Univariately we have a highly significant association, so I put it in the multivariable model. In the multivariable model, we still have high significance but the CI for the OR is really wide. $\endgroup$– user213352Jul 2, 2018 at 18:38
-
$\begingroup$ So disease $x$ is an extremely good predictor of mortality? How many patients did you have alive in total? How was the information collected (regarding independence, was it more or less likely to find a combination death-disease due to the sampling)? What would have been the expected number of alive patients in the desease x group? $\endgroup$– Sextus EmpiricusJul 2, 2018 at 18:57
-
$\begingroup$ Over 200 patients alive and only 41 dead. Information was collected independently (no sampling issues). $\endgroup$– user213352Jul 2, 2018 at 19:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Logistic regression has a known issue with overadjustment. Estimates of the odds ratio become biased in smaller samples: the OR tends toward it's square. So an OR of 2 tends to 4 as the statistical power to detect that odds ratio diminishes. The CI widens with this tendency as well so that the interval estimates are anticonservative for replications of the design. See, also, a relevant answer here.
The amount of information you can glean from this analysis is somewhat limited and disappointing:
- This feature probably has prognostic value
- The analysis is not adequately powered to give precise estimates.
- The risk prediction model will tend to be miscalibrated by underestimating risk in people not afflicted with this exposure.
- The risk model which omits this factor will estimate a reduced model precisely, but has the subsequent omitted variable bias which will have unanticipated and limited generalizabiity.
The efficient solution is re-estimate sample size, and possibly use targetted, two-phase sampling to be able to reliably estimate the impact of this exposure in the context of a multivariate model.
