Tweedie Dispersion Parameter Estimation Methods

Question

In the book Generalized Linear Models with Examples in R - Dunn and Smyth, in Chapter 6.8, it is recommended to use the Pearson estimator of the dispersion - "This makes the Pearson estimator the most robust estimator, in the sense it relies on fewest assumptions. For this reason the glm() function in R uses the Pearson estimator for $\phi$ by default".

However, in the tweedie.profile () function from the tweedie package in R, it is not even given as an option for the dispersion estimation, rather the mean deviance and MLE are suggested (with MLE as default).

What is the reason for that? why would the Pearson estimate will be better when fitting a GLM, but not used when selecting the power parameter of Tweedie distribution?

Answer 1

Tweedie generalized linear models assume a mean-variance relationship with variance power $p$, defined by $$E(y_i)=\mu_i$$ and $${\rm var}(y_i)=\phi \mu_i^p$$ where $y_i$ is the $i$th observation, $\mu_i$ is the expected value, $\phi$ is the dispersion and $p$ is the mean-variance power parameter, also called the Tweedie index parameter.

The purpose of the tweedie.profile function is to estimate $p$ by profile maximum likelihood. The dispersion $\phi$ is a nuisance parameter in the profile likelihood, as are the glm regression coefficients. It is a requirement of the profile likelihood method in statistics that all nuisance parameters be set to their maximum likelihood estimates. It would be invalid to use the Pearson estimator for $\phi$ is this context because then the resulting likelihood for $p$ would not be a profile likelihood and differences in log-likelihoods would not have their usual meaning.

We recommend that you use tweedie.profile to guide your choice of the Tweedie index parameter. Then you can fit a generalized linear model treating $p$ as a known parameter, for example by:

library(statmod)
fit <- glm(y~x, family=tweedie(var.power=p, link.power=0))
summary(fit)

In the above code, the summary function in R will by default use the Pearson estimator for $\phi$. The use of the Pearson estimator is not specific to Tweedie distributions, but is a generic property of the summary.glm function in R. The fact that tweedie.profile and summary.glm use different estimators for $\phi$ does not cause any problems for the process.

As hinted at in Section 6.8 of our book (Dunn & Smyth, 2018), I would ideally prefer to use modified profile likelihood for both $p$ and $\phi$, but that is difficult to implement. The above process is much simpler and will still give good results.

I see that I answered a related question from another poster 3 years ago: GLM Tweedie dispersion parameter.

Reference

Dunn PK, Smyth GK (2018). Generalized linear models with examples in R. Springer, New York, NY.