R: Coping with bad model fit convergence during bootstrap

Question

I'm running into trouble while checking the robustness of a GLM using the bootstrap procedures in R. Basically I want to check whether the estimated coefficient estimates of an existing GLM that I've built before are consistent and robust for my data.

The model fit is usually pretty consistent and nice, but every once in a while (0.5% of the cases), the model doesn't convergence for some reason or another. The end result is that I can't get decent bootstrap statistics and so forth. Is there a known solution for this in literature? Here's what I tried in R:

statfun <- function(formula, data, indices) {
    d <- data[unique(indices),] # allows boot to select sample
    fit <- glm(formula, data=d,family="binomial")
    if(!fit$converged) {
        return(NULL) #ignore this fit? this doesn't work ...
    }
    return(coef(fit))
}
# bootstrapping with 1000 replications
results <- boot(data=df, statistic=statfun,R=1000, formula=form)

Update: This data contains about 4500 rows, 11 columns, 3 of which are dummies and thus have repeated entries quite often (2 covar and the dependent var). I checked whether in any sample iteration those dummies are exclusively restricted to one value, to which the answer is negative.

Answer 1

Two sources of your problem come to mind.

First, there can be issues with stability of convergence using glm. There is a package glm2 in R that I believe uses an adaptive step-size choice to improve convergence. That might solve your problem.

There's also a possibility that your model is close to providing perfect separation between 0/1 values of your dependent variable. This can happen even with surprisingly large ratios of cases to predictors.

The warnings generated for the bootstrap samples that are giving you the problems should help distinguish these possibilities. Your statfun function could be modified to save the indices so that you can examine the particular samples giving the problem, or export the particular warnings/errors that are generated.

When you go back to bootstrapping your data, do not remove the duplicate rows. Those are critical to proper bootstrapping, which is deliberately done with replacement. Remember that you are trying to check the robustness of your GLM with respect to the underlying population, and your data set provides the best available estimate of your underlying population. If your problem is arising from perfect separation in 1 out of 200 sample from that population (instead of from numerical convergence issues), then you have to consider how robust your present model is and consider approaches like penalized regression suggested in the pages linked above.