3
$\begingroup$

I am using ?bam in mgcv 1.8-24, and when I'm using the negative binomial family I sometimes get a warning that says:

In estimate.theta(theta, family, y, mu, scale = scale1, wt = G$w, :  
step failure in theta estimation

The results from the model fit still seem reasonable. I can often get the warning to go away just by changing k slightly for my smooth term (I am using mostly parametric terms). After adjusting k and getting rid of the warning, the results look the same.

I haven't been able to find any description of this warning. What is a "step failure", and is it something that I need to worry about?

Improve this question
$\endgroup$
4
  • $\begingroup$ This question is probably still bound up in the implementation in the R code, but after the edit, it seems to me it is a legitimate statistical question. I'm voting to reopen. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 5, 2019 at 20:03
  • $\begingroup$ Can you provide a small example dataset & simple code that will reproduce this error? Although Cross Validated isn't an R tech support site (hence the prior hold), it may be necessary for people to determine the answer. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 5, 2019 at 20:10
  • $\begingroup$ Well, I've found that the warning goes away when I don't use discretization (an option that makes fitting faster), which makes me think that it IS more of a software issue and less of a statistical question. If anyone knows the details of the algorithm and wants to share why this might be, that would be great, but otherwise this can be closed. $\endgroup$
    – dante
    Commented Apr 17, 2019 at 2:04
  • $\begingroup$ It's up to you, @dante. If you want, you can delete the thread yourself. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 17, 2019 at 4:19

1 Answer 1

Reset to default
0
$\begingroup$

I don't have enough reputation to comment, but after hitting this exact error I can confirm that it seems to be a software issue. Setting discrete=FALSE in the bam gives identical results and residuals to discrete=TRUE but without the error. Hope that helps the next person who stumbles across the same issue!

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ It's not a software issue; you are fitting with different data depending on the value of discrete and this warning can occur outside of fitting with bam(). It indicates a problem during optimization of one of the parameters (no steps the algorithm took to walk downhill to hopefully the min of the penalised log likelihood actually resulted in a reduction in that criterion but that convergence criteria weren't met at the current estimates of model coefs) and likely that there is some issue or incompatibility between the data and the model you are fitting to them. $\endgroup$
    – Gavin Simpson
    Commented May 23, 2022 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.