Questions tagged [tweedie-distribution]

A family of distributions from the exponential dispersion family with a power-law mean-variance relationship. For power $p$ between 1 and 2, it is a compound Poisson-Gamma distribution that has point mass at zero and is continuous on positive numbers.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Regression with zero-inflated outcome

I am trying to fit and tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not ...
1
vote
0answers
27 views

How to estimate power index for tweedie glm on large data

I've went through the tweedie chapter in the book Generalized Linear Models with Examples in R and it seems that the function tweedie.profile is used to estimate the power index for tweedie glm. ...
2
votes
1answer
35 views

What does it mean when MAE improved while RMSE worsened?

I am comparing two models: One is a black box that I cannot understand, the other is a GLM. How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
0
votes
1answer
74 views

GLM Tweedie dispersion parameter

I am running a tweedie regression and I need to get the dispersion parameter(phi). Here a simple R code: ...
3
votes
1answer
76 views

Overdispersion vs Tweedie

I am dealing with data that could possibly be overdispersed and I am looking at fitting a GLM with a quasi distribution. As far as I understand, when we fit a glm ...
1
vote
0answers
46 views

Replicating a Tweedie corrected experiment from Computer Age Statistical Inference

I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411. In this experiment 100 datasets are populated normal random ...
0
votes
0answers
31 views

What model is appropriate when a non-negative, continuous dependent variable has frequent zero-values because of limited data?

I'm modeling how likely a song is to occur in a playlist with a particular title. I can calculate a simple probability based on my current data, but it's highly zero-inflated because my data is ...
0
votes
1answer
191 views

Tweedie distribution without zeroes

Recently I've found the Tweedie distribution useful for modelling my plant shoot weight data in glmmTMB. I started using it because my shoot weight datasets have many zeroes, and it has yielded better ...
4
votes
1answer
382 views

Chi Square vs F Tests for GLM Model Comparisons

I've been creating some models in R using glm() and rxGlm(). I'm experienced in building GLMs but my memory of some of the ...
0
votes
0answers
46 views

How do I properly choose a value for 'phi'

How does one choose the proper value for the phi parameter in a random tweedie sample. I'm trying to properly choose the correct value for the 'phi' parameter for ...
3
votes
2answers
523 views

Does the dependent variable in a GLM have to be transformed before running the model or does the model do it?

I'm trying to fit a Tweedie model with statsmodels and was wondering if I have to transform the dependent variable before I run the model or if ...
2
votes
0answers
100 views

SAS and R gives different standard error when fitting Tweedie model

I would like to fit a Tweedie GLM to the data. However, even though PROC GENMOD from SAS and glm() from R gives the same coefficient estimates, they give quite different standard error. What it the ...
0
votes
1answer
275 views

h2o glm tweedie for categorical variables?

To build a tweedie glm for categorical variables, the document suggested that I can use data['variable_name'].asfactor(). However, in the model output, there is ...
2
votes
0answers
47 views

Best way to test the difference in means for Tweedie distributions?

What is the best way to test the difference in means for two samples of data that follow a Tweedie distribution? (zero inflated, tweedie power parameter ~1.7) The data represents the number of ...
2
votes
1answer
1k views

Weak learners for XGBoost with Tweedie distribution

Could you please explain what are the standard weak learners for XGBoost when the objective parameter equals reg:tweedie? Are they GLMs (with Tweedie distribution of dependent variable) on all ...
3
votes
2answers
524 views

Non normal residuals for Tweedie GLM

I am using Tweedie GLM as my data contains exact zeroes. However, my stats is weak and want to confirm a few things. Does Tweedie GLM assume normality of residuals? Is shapiro.test() the way for ...
1
vote
0answers
344 views

Tweedie GLM and model selection

I have run a tweedie glm since my data is zero skewed. How can I use Akaike weights and evidence ratio to choose from models the better one. Please guide. I had earlier run models with glm but with ...
1
vote
0answers
126 views

R.sq in gam model (mgcv)

first question here. I'm constructing GAMs for multispecies avian survey data, and running into some odd results in the calculation of R.sq in the summary() command. For one of my species, when i use ...
0
votes
1answer
205 views

Fitting a Poisson-Tweedie model with random effects

I'd like to fit a Poisson-Tweedie model with random effects. I'm wondering if it is possible to do it with the mcglm package. I'm following the work of Bonat et al (2017): "Extended Poisson-Tweedie: ...
2
votes
0answers
53 views

Tweedie index parameters are restricted: why?

The Tweedie distribution have variance like var(y) = $\phi \mu^p$ for any real p not between 0 and 1. I read in many places (even wikipedia) that p can take any real value except between 0 and 1. ...
5
votes
0answers
858 views

generalized linear mixed-effects models R^2 and the tweedie distribution [closed]

I am modelling data exhibiting a tweedie distribution in R using glmer (package lme4). To compare the models I would like to use the AIC and R^2. I have a couple of question on this (example code at ...
7
votes
1answer
6k views

Given a GLM using Tweedie, how do I find the coefficients?

Let $Y$ be a random variable that obeys the Tweedie distribution for parameter $\alpha = 1.1$. Let the link function be the natural log. Assume that we have a database of numbers of the form $(y_1, ...
5
votes
1answer
526 views

Skewness of Tweedie distribution

Tweedie distributions are a family of distributions from the exponential dispersion family that have power-law mean-variance relationship: \begin{align} \mathbb E[X] &= \mu \\ \operatorname{Var}[...
0
votes
0answers
763 views

deviance for zero-inflated compound poisson model, continuous data (R)

I am looking at homicide rates per US county in a given 3-yr period as a DV, against several demographic and political measures as IV. I have A LOT of zeros for the DV, and these are presumably real ...
2
votes
2answers
1k views

How do you calculate the Tweedie prediction based on model coefficients?

What is the formula to calculate the prediction for a Tweedie distribution using model coefficients? I am trying to manually calculate the prediction. Below is my attempt at reproducible code. <...
0
votes
1answer
177 views

Skewed Response Variable a lot zeroes

I am trying to build a non-parametric model to predict pure premium of an insurance policy. Pure premium is simply the expected claim amount of an insurance policy.The problem is, most insurance ...
1
vote
0answers
269 views

CDF of a Tweedie Distribution

The R package tweedie provides a way to estiamate the density and CDF of Tweedie distribution (using dtweedie and ptweedie, ...
0
votes
0answers
900 views

What's the reason for getting NaN values when predicting Tweedie GLM response?

Question is in the title. Not asking an R question, but the NaN result was in R. I just wonder why this happens for Tweedie GLMs. Example code in R, where ...
0
votes
1answer
515 views

When fitting Tweedie GLM in SAS, should I use the scale option?

In SAS proc genmod with dist=tweedie, when would you use scale=pearson or ...
16
votes
1answer
17k views

How to model non-negative zero-inflated continuous data?

I'm currently trying to apply a linear model (family = gaussian) to an indicator of biodiversity that cannot take values lower than zero, is zero-inflated and is ...
15
votes
3answers
4k views

Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?

A Tweedie distribution can model skewed data with a point mass at zero when the parameter $p$ (exponent in the mean-variance relationship) is between 1 and 2. Similarly a zero-inflated (whether ...
1
vote
1answer
723 views

A model for non-negative data with many zeros: pros and cons of Tweedie GLM

I analyze technical measurement data with the aim of developing a forecasting model. The data is given as a non-negative time series (data per hour). The data looks quite wild and contain many zeros. ...
4
votes
0answers
671 views

How a tweedie glm handles an offset?

I am trying to fit a model with a glm using a tweedie family. I use a index parameter p between 1 and 2 to get a compound Poisson Gamma distribution to fit my data. But I want to use an offset only on ...
1
vote
1answer
2k views

GLM model selection using AICc with Tweedie distribution

I have two questions regarding use of Tweedie GLM in R. I am new in using this distribution and despite a thorough search on different forums, I could not find my answers. I am now running several ...
2
votes
0answers
223 views

Compound Poisson Distributions: When, Why, and How To Split the Problem

I've just stumbled upon the Compound Poisson Distribution (CPD) and it seems to be precisely what I need. For the purposes of this post, let's suppose I have a store that sells many items of ...
11
votes
1answer
3k views

What is the canonical link function for a Tweedie GLM?

I was just introduced to the Tweedie distribution (see this or this) but I'm having a hard time finding what the link function is for a Tweedie generalized linear model. Thoughts?
2
votes
1answer
6k views

glm.fit: algorithm did not converge -Tweedie

I'm trying to estimate $p$ in tweedie regression, but I got the following message: glm.fit: algorithm did not converge I'm using public data from "GLMs for insurance data" book by Piet de Jong, ...
5
votes
1answer
1k views

Gamma vs tweedie distribution for large productivity dataset

I'm running some GAMs using the mgcv R package on a dataset with ~8.5k observations, where productivity is the response and environmental conditions are the covariates. However I am unsure of which ...
4
votes
0answers
2k views

Tweedie P Value Interpretation

From Wikipedia (http://en.wikipedia.org/wiki/Tweedie_distribution) we know that The Tweedie distributions include a number of familiar distributions as well as some unusual ones, each being ...
12
votes
2answers
10k views

Possible to evaluate GLM in Python/scikit-learn using the Poisson, Gamma, or Tweedie distributions as the family for the error distribution?

Trying to learn some Python and Sklearn, but for my work I need to run regressions that use error distributions from the Poisson, Gamma, and especially Tweedie families. I don't see anything in the ...
4
votes
0answers
907 views

R codes for Tweedie compound Poisson gamma [closed]

I have modeled claim frequency data using Poisson regression and claim amount using gamma. I have seen the Tweedie compound Poisson gamma distribution used to model aggregate claim data. I am quite ...