Is it valid to run logistic regression on zero inflated data where the response variable is dead/alive?
$\begingroup$
$\endgroup$
9
-
$\begingroup$ What is zero-inflated in this case, the response variable? $\endgroup$– DaveCommented Aug 12, 2020 at 0:38
-
$\begingroup$ There’s a lot of alives and few dead. $\endgroup$– user293790Commented Aug 12, 2020 at 0:39
-
$\begingroup$ Rarely every does death happens $\endgroup$– user293790Commented Aug 12, 2020 at 0:40
-
$\begingroup$ If someone suggests poisson regression, I’ll say no because I’ll have to use quasi poisson with my zero inflated data, or use NBGLMM which I don’t want to do. $\endgroup$– user293790Commented Aug 12, 2020 at 0:42
-
1$\begingroup$ You don’t have zero-inflated data. You have imbalanced classes. Poisson regression does not come into play. Poisson would be when the response is a count. Your response is a category that you happen to code as 0 and 1, but those numbers have no meaning. (You could flip which class corresponds time which number without changing much of anything.) $\endgroup$– DaveCommented Aug 12, 2020 at 0:50
|
Show 4 more comments
1 Answer
$\begingroup$
$\endgroup$
At the risk of misinterpreting, it seems that the question really is about whether logistic regression is valid when the 0-1 (dead/alive) variable is far from a 50-50 split. The answer is yes, you can use logistic regression with any split, even 99.99 - 00.01. Logistic regression estimates the conditional distribution (Bernoulli) of the 0-1 variable as a function of the predictors, and this distribution can have any probabilities - (.5,5), (.7,.3), (.0001,.9999) etc.
From a comment, I guess one of the predictors is time.
I once worked for a burn unit on a similar problem: Estimate the distribution of burn mortality (dead/alive) as a function of age, % burn, and time (year). The specific interest was in particular time points where new antibacterial treatments were introduced, and the effect on mortality, holding %burn and age constant. We used a kind of moving average method to estimate the time effect, so we did not assume any particular constrained function form (eg linear) for the time variable. Using this method, we were able to find very dramatic effects of certain interventions that saved many lives. The research is in a technical report of the US Army Institute for Surgical Research in San Antonio, Texas.
A caveat about the imbalance is that with extreme imbalance, especially coupled with small sample sizes, there can be no maximum likelihood estimates. This is called "quasi-separation," and if it happens, the computer will alert you to it, so you don't need to worry about it up front. If it happens, you'll have to drop a covariate or two; and if you don't want to do that you can use Bayesian methods.
answered Aug 18, 2020 at 11:02