# Tweedie p parameter Interpretation

From Wikipedia (http://en.wikipedia.org/wiki/Tweedie_distribution) we know that

The Tweedie distributions include a number of familiar distributions as well as some unusual ones, each being specified by the domain of the index parameter. We have the
normal distribution, p = 0,
Poisson distribution, p = 1,
compound Poisson–gamma distribution, 1 < p < 2,
gamma distribution, p = 2,
positive stable distributions, 2 < p < 3,
inverse Gaussian distribution, p = 3,
positive stable distributions, p > 3, and
extreme stable distributions, p = ∞.

For 0 < p < 1 no Tweedie model exists.

My question is can we determine significance from the optimal $p$ if it is between 1-2. For instance, if we say that Poisson controls the frequency in which our $Y$ appears and Gamma controls the size of our $Y$ if it appears and if the log-likelihood of the model has an optimal value for $p$ of 1.1 then is it fair to say that the data is driven more by frequency than size and vice versa?

so for example, if you run this code:

set.seed(1)
x <- rep(1:100,times=10)
freq <- sapply(x,function(arg) rpois(1,.25*arg))
size <- sapply(x,function(arg) rgamma(1,100*arg))
y <- freq*size
df <- data.frame(cbind(x,y))

tweediep <- tweedie.profile(y~x, link.power = 0, data = df,
do.smooth = FALSE, do.plot = TRUE)
tweediep$xi.max  I get the result 1.3. Does that mean that this Y is more "driven by" frequency than severity? Does it mean something else altogether? Is there nothing I can discern from this estimated value of the parameter$p$? Edit: This is not a complete answer so I'm posting it as an edit and clarification rather than an answer to my own question. I have attempted to anecdotally answer my own question via a limited experiment. Specifically, I want to create dummy data and a "frequency driven" and "severity driven" model from this data and observe the results to better understand the$p$parameter. set.seed(1) #creating random variables x1<-rep(1:5,20) x2<-(runif(100))*10 x3<-rpois(100,5) x4<-rweibull(100,2,5) #creating parameters based off a linear combination of variables plus a noise term - all exponentiated fvars<-exp((x1+2*x2+.5*x3-x4+3*runif(1))/10)/5 svars<-exp((x1+2*x2+.5*x3-x4+3*runif(1))/5) summary(fvars) summary(svars) #freq is the number of occurances, sev is the size of the occurances freq<-rpois(100,fvars) sev<-rgamma(100,scale=svars,shape=1) summary(freq) summary(sev) results1<-freq*sev summary(results1) #load library if needed library(tweedie) #store all the data in a single data frame df1<-data.frame(cbind(x1,x2,x3,x4,results1))  This is my "base model". I now find the optimal parameter$p$. tweediep<-tweedie.profile(formula = results1~x1+x2+x3+x4,link.power = 0,do.smooth = TRUE,do.plot = TRUE) tweediep$xi.max


I get a result of 1.616327 . I went ahead and found the results model coefficients. Strictly for the purposes of answering my question, I don't believe actually fitting the glm is necessary but perhaps the diagnostics of the glm will lead to more insight.

baseGLM<-glm(results1~x1+x2+x3+x4,data=df1,family=tweedie(var.power = tweediep$xi.max,link.power = 0)) summary(baseGLM)  It's possible that the model isn't "fit well enough" for this type of analysis to say anything meaningful, but to continue this line of reasoning, let's try fitting a "frequency driven model". #freq driven freq<-rpois(100,fvars*3) sev<-rgamma(100,scale=svars/4,shape=1) summary(freq) summary(sev) results2<-freq*sev summary(results2) df1<-data.frame(cbind(df1,results2)) tweediep2<-tweedie.profile(formula = results2~x1+x2+x3+x4,link.power = 0,do.smooth = TRUE,do.plot = TRUE) tweediep2$xi.max


here our $p$ parameter value is 1.718367 which is higher than our base value. Again, we fit the model.

GLMfreqDriven<-glm(results2~x1+x2+x3+x4,data=df1,family=tweedie(var.power = tweediep2$xi.max,link.power = 0)) summary(GLMfreqDriven)  Finally, let's make our model more severity driven. #severity driven freq<-rpois(100,fvars*.4) sev<-rgamma(100,scale=svars*5,shape=1) summary(freq) summary(sev) results3<-freq*sev summary(results3) df1<-data.frame(cbind(df1,results3)) tweediep3<-tweedie.profile(formula = results3~x1+x2+x3+x4,link.power = 0,do.smooth = TRUE,do.plot = TRUE) tweediep3$xi.max


Here our $p$ parameter is 1.591937.

This is the opposite of my initial hypothesis. I originally thought that the more frequency driven a model is, the closer the p parameter would be to 1 (since 1 is a true Poisson model), and the more severity driven the model is, closer the p parameter is to 2. In my, admittedly limited, experiment the opposite was true. The Poisson distribution does allow zeros as a response variable. Perhaps the p parameter has nothing to do with "frequency driven" or "severity driven", but is strictly a function of the number of zeros in the data (for p parameter values between 1 and 2).

One possible limitation I thought of with respect to my experiment design is that would be interesting to see how the results change if I force the results1, results2, and results3 variables to all have the same mean. The data here was fit arbitrarily. Several of instances where I am dividing or multiplying by a constant, I am doing so to get the data in such a way that the algorithm converges without error. I would be more confident if the results (1.61, 1.72, 1.59) were more spread out.

In conclusion, I can see possible explanations for results of my limited experiment, but would really be appreciative if someone could further validate and explain how the $p$ parameter is affected by the data. Any discussion as to the differences between creating separate frequency and severity models versus creating compound Poisson-gamma models may also prove to be very insightful.

Thank you

The Generalized Linear Models with Examples in R book by Peter Dunn and Gordon Smyth contains an illuminating discussion of Tweedie distributions.

If I maybe so blunt, and summarize your excellent question:

What is the relation of $$p$$ and the underlying Poisson-Gamma model for a Tweedie distribution with $$1 < p < 2$$?

As you already note in your question, the Tweedie distribution with $$1 < p < 2$$ can be understood as a Poisson-Gamma model. To make it more concrete what this means, let's assume that

$$N \sim \text{Pois}(\lambda^{*})$$ and $$z_i \sim \text{Gam}(\mu^{*}, \phi^{*})$$ the observed $$y$$ is $$y = \sum_{i = 1}^{N}{z_i}.$$ Dunn & Smyth give an example for this model, where $$N$$ is the number of insurance claims and $$z_i$$ is the average cost for each claim. In that case the model would describe the total insurance payout.

The relation of $$p$$ to the parameters of the Poisson and Gamma distribution is

\begin{equation} \begin{aligned} \lambda^{*} &= \frac{\mu^{2-p}}{\phi (2-p)} \\ \mu^{*} &= (2 - p)\phi\mu^{p - 1} \\ \phi^{*} &= (2 - p)(p - 1) \phi^2\mu^{2(p - 1)}, \end{aligned} \end{equation} where $$\mu$$ and $$\phi$$ are the mean and overdispersion parameters from the generalized linear model definition.