I've been asked to fit a ZeroInflatedPoisson model on a dataset to predict Y (count data) for an assignment. First, I did this manually:
- Create a binary variable (Y_IND) based on Y where Y_IND = 0 if Y = 0, and 1 if Y >=1.
- Fit a statsmodels Logistic Regression model using X variables to predict the binary variable Y_IND with no problem.
- Fit a statsmodel Poisson Regression model on the subset of data where Y_IND = 1, using X variables to predict Y, which also worked without issue.
However, when I tried to fit a model using statsmodels ZeroInflatedPoisson to predict Y, I received error messages stating:
"ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals ConvergenceWarning)"
This is my code:
sm.ZeroInflatedPoisson(endog=y, exog=X, exog_infl=X, inflation='logit').fit(maxiter=100)
I assume the warning message is referring to the Logistic Regression step, but I'm not sure why since it was able to fit a Logistic Regression model independently on the same dataset.
Is it unable to converge because it is trying predict Y instead of the binary Y_IND? If so, how can I get around this issue?
model.fit(method="nm", maxiter=5000)as a more robust optimizer. $\endgroup$