2
$\begingroup$

I'm running into trouble while checking the robustness of a GLM using the bootstrap procedures in R. Basically I want to check whether the estimated coefficient estimates of an existing GLM that I've built before are consistent and robust for my data.

The model fit is usually pretty consistent and nice, but every once in a while (0.5% of the cases), the model doesn't convergence for some reason or another. The end result is that I can't get decent bootstrap statistics and so forth. Is there a known solution for this in literature? Here's what I tried in R:

statfun <- function(formula, data, indices) {
    d <- data[unique(indices),] # allows boot to select sample
    fit <- glm(formula, data=d,family="binomial")
    if(!fit$converged) {
        return(NULL) #ignore this fit? this doesn't work ...
    }
    return(coef(fit))
}
# bootstrapping with 1000 replications
results <- boot(data=df, statistic=statfun,R=1000, formula=form)

Update: This data contains about 4500 rows, 11 columns, 3 of which are dummies and thus have repeated entries quite often (2 covar and the dependent var). I checked whether in any sample iteration those dummies are exclusively restricted to one value, to which the answer is negative.

Improve this question
$\endgroup$
2
  • $\begingroup$ How many rows are in your data set? And do many of them have identical values for all the variables in your model? $\endgroup$
    – EdM
    Jul 3, 2015 at 15:48
  • $\begingroup$ Updated the question to include dataset info, I did notice "boot" produces duplication of rows (so the indices array is not unique). I tried removing duplicates as you can see in the above code, but to no avail. $\endgroup$
    – ciri
    Jul 3, 2015 at 16:01

1 Answer 1

Reset to default
3
$\begingroup$

Two sources of your problem come to mind.

First, there can be issues with stability of convergence using glm. There is a package glm2 in R that I believe uses an adaptive step-size choice to improve convergence. That might solve your problem.

There's also a possibility that your model is close to providing perfect separation between 0/1 values of your dependent variable. This can happen even with surprisingly large ratios of cases to predictors.

The warnings generated for the bootstrap samples that are giving you the problems should help distinguish these possibilities. Your statfun function could be modified to save the indices so that you can examine the particular samples giving the problem, or export the particular warnings/errors that are generated.

When you go back to bootstrapping your data, do not remove the duplicate rows. Those are critical to proper bootstrapping, which is deliberately done with replacement. Remember that you are trying to check the robustness of your GLM with respect to the underlying population, and your data set provides the best available estimate of your underlying population. If your problem is arising from perfect separation in 1 out of 200 sample from that population (instead of from numerical convergence issues), then you have to consider how robust your present model is and consider approaches like penalized regression suggested in the pages linked above.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ glm2 solved the problem for me, thank you very much EdM! $\endgroup$
    – ciri
    Jul 3, 2015 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.