Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
When running and gam model and this warning message pops up, what does this mean and what is advised? A simple increase in knots (even by 1) seems to remove this warning message.
"Warning message: In estimate.theta(theta, family, y, mu, scale = scale1, wt = weights, : step failure in theta estimation"
Step failure is reported when the algorithm is unable to take a step in the parameter space that would result in an improvement in the optimization criterion. In this case it is having trouble moving in the direction that the derivative of $\theta$ with respect to the other model parameters indicates will improve the criterion. So there is a quandry; everything is telling the algorithm that it hasn't reached convergence and that it could actually do something to improve the fit, but when it tries to do that it fails to find a step down the surface of the criterion that actually results in improved fit.
Basically the algorithm used to estimate the model failed to converge and it is reporting where the convergence problem raised its head.
Other questions asking about this specific issue are:
Warnings like this are often accompanied by another note that you should check the model carefully. That would involve looking at the estimated value of $\theta$ (use family(model)$getTheta()) and seeing what that tells you, plus usual model checks. Checking if there are unusual (and influential) observations (via the partial residuals) might be one way to diagnose the issue.
The output from gam.check() that is printed to the console also contains some useful information with respect to the optimization problem, which might also be useful, but you'll likely need to be pretty familiar with {mgcv} or optimization in general for this to make sense.
Changing the model by increasing k by 1 unit changes the model parameter space and it may result in a surface that isn't a weird as the one encountered originally, and the model no longer has issues finding the minimum of the optimization criterion.
Why does changing the basis size by such a small amount have such a noticeable effect? This could be simply due to span of functions in the basis with dimension k + 1 being (much?) larger with respect to certain complex functions than the span of the same function in a basis with dimension k.
