Questions tagged [tweedie-distribution]

A family of distributions from the exponential dispersion family with a power-law mean-variance relationship. For power $p$ between 1 and 2, it is a compound Poisson-Gamma distribution that has point mass at zero and is continuous on positive numbers.

Filter by
Sorted by
Tagged with
1
vote
0answers
49 views

Understanding the Tweedie Distribution

I am trying to better understand the basic details and properties about the Tweedie Probability Distribution. The Tweedie Probability Distribution is a bimodal probability distribution function which ...
1
vote
0answers
77 views

why is gradient boosting machine based algorithm nonparametric although they have a loss function?

I am dealing with very right-skewed distribution of y which seems to follow a tweedie distribution. and I found out it would give a higher performance to change lgbm's loss function to a "tweedie&...
3
votes
1answer
147 views

Interpreting tweedie-location-scale result from R mgcv package

I came across the Tweedie-location-scale family function in R (twlss function from mgcv). It looks appealing to model my data. I understand the general concepts is that "twlss" family allows ...
0
votes
0answers
95 views

Tweedie distributions for large dataset, power parameter for different density peaks and random effects

I have a large dataset (343750 variables and 151 observations) and I want to model each variable as a response one in order to know if it can be explained by the group of patients and the gene ...
2
votes
1answer
141 views

Tweedie GLM, what var.power should I chose?

Im trying to do a GLM on my data about bats to see how different variables affect bat activity on 8 species. Orignally my data was count data, but because of hardware difficulties I had to divide the ...
0
votes
0answers
58 views

Regression model for extremely right-skewed and zero-inflated data

Background I'm trying to predict the cost of road network performance (a scalar) from a set of 142 predictors -- 71 describe the fragility of each bridge in the road network and 71 describe the ground-...
0
votes
0answers
128 views

zero-inflated right skewed data - cpglm output

So I have a right skewed zero inflated dependent variable and I wish to run a model with that. It corresponds to a value of association between two words (so if 0 words are not associated). and I want ...
0
votes
0answers
73 views

How to handle exces zero's in a continuous (multilevel) outcome: two-part model or Tweedie distribution?

I am trying to fit a hierarchical model on a continuous outcome that has excess zero's. This is what my data looks like: The formula for both the main part as well a possible zero-part of the model ...
1
vote
0answers
166 views

Comparing the fit of Tweedie and gamma hurdle models

I am comparing several modeling approaches to semi-continuous data (many exact zeros and continuous positive cost outcomes) to assess the effect of the main predictor "disease" on cost. The ...
1
vote
1answer
560 views

Python statsmodels constant and error using Tweedie distribution

I'm following a paper described in the link which models rainfall. Due to a cyclical nature the authors use a model equation (53): $$ log(\mu_{i}) = a_{0} + a_{1} sin (\frac{2\pi i}{365}) + a_{2} ...
10
votes
1answer
2k views

What is use of Tweedie or poisson loss/objective function in XGboost and Deep learning models

I am looking at few competitions in kaggle where people used tweedie loss or poisson loss as objective function for forecasting sales or predicting insurance claims. Can someone please explain the ...
3
votes
0answers
448 views

How do I choose the correct power in a tweedie GLM?

I'm developing a GLM with a tweedie family distribution. How do I choose P correctly and how much does it matter? I know it has to be between 1 and 2, and when I try different values, I don't see an ...
2
votes
0answers
357 views

XGBoost/LightGBM underpredicts for some categories but is otherwise good

I am trying to predict the revenue that a click generates on an average (revenue per click). There are K different items that get shown to different users. My training data consist of clicks. A ...
1
vote
0answers
85 views

Comparing a hurdle model with a "direct" model

I am building xgboost models for prediction of insurance risk, the risk being assumed to follow a tweedie distribution with tweedie variance power between 1 and 2 (https://en.wikipedia.org/wiki/...
0
votes
1answer
199 views

How do I manually predict Tweedie GLM

I have a simple Tweedie model in GLM, or $Y_i=\beta_0+\beta_1x_1$. As usual, $Y$ is the output and $x_i$ is its dependent variable. Suppose that $p>1$, my link function would be $\frac{\mu^{1-p}}{1-...
0
votes
0answers
21 views

R: tweedie.dev function gives Inf

I have a question regarding the function tweedie.dev(y, mu, power) from the tweedie package. When I run this function on some data I have generated I get Inf where y = 0 (I use power = 1.5). Does this ...
0
votes
1answer
99 views

Identifying the best distribution to this data?

I'm trying to fit an appropriate distribution to a data with 216 values and estimate parameters. From Cullen and Frey graph, it looks like lognormal could be a good fit. From q-q plot, Weibull seems ...
1
vote
0answers
67 views

Generate random variates and cdf from Tweedie distribution

I have to generate some random variates from a Tweedie distribution with the following parameters: $p = 1$ $\mu = \bar{\mu}$ $\phi = \bar{\phi}$ I do not have direct access to a function that is able ...
2
votes
0answers
424 views

How to estimate power index for tweedie glm on large data

I've went through the tweedie chapter in the book Generalized Linear Models with Examples in R and it seems that the function tweedie.profile is used to estimate the power index for tweedie glm. ...
2
votes
1answer
80 views

What does it mean when MAE improved while RMSE worsened?

I am comparing two models: One is a black box that I cannot understand, the other is a GLM. How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
0
votes
1answer
914 views

GLM Tweedie dispersion parameter

I am running a tweedie regression and I need to get the dispersion parameter(phi). Here a simple R code: ...
3
votes
1answer
439 views

Overdispersion vs Tweedie

I am dealing with data that could possibly be overdispersed and I am looking at fitting a GLM with a quasi distribution. As far as I understand, when we fit a glm ...
2
votes
0answers
72 views

Replicating a Tweedie corrected experiment from Computer Age Statistical Inference

I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411. In this experiment 100 datasets are populated normal random ...
1
vote
1answer
745 views

Tweedie distribution without zeroes

Recently I've found the Tweedie distribution useful for modelling my plant shoot weight data in glmmTMB. I started using it because my shoot weight datasets have many zeroes, and it has yielded better ...
6
votes
1answer
2k views

Chi Square vs F Tests for GLM Model Comparisons

I've been creating some models in R using glm() and rxGlm(). I'm experienced in building GLMs but my memory of some of the ...
0
votes
0answers
72 views

How do I properly choose a value for 'phi'

How does one choose the proper value for the phi parameter in a random tweedie sample. I'm trying to properly choose the correct value for the 'phi' parameter for ...
4
votes
2answers
1k views

Does the dependent variable in a GLM have to be transformed before running the model or does the model do it?

I'm trying to fit a Tweedie model with statsmodels and was wondering if I have to transform the dependent variable before I run the model or if ...
2
votes
0answers
176 views

SAS and R gives different standard error when fitting Tweedie model

I would like to fit a Tweedie GLM to the data. However, even though PROC GENMOD from SAS and glm() from R gives the same coefficient estimates, they give quite different standard error. What it the ...
0
votes
1answer
868 views

h2o glm tweedie for categorical variables?

To build a tweedie glm for categorical variables, the document suggested that I can use data['variable_name'].asfactor(). However, in the model output, there is ...
2
votes
0answers
118 views

Best way to test the difference in means for Tweedie distributions?

What is the best way to test the difference in means for two samples of data that follow a Tweedie distribution? (zero inflated, tweedie power parameter ~1.7) The data represents the number of ...
3
votes
1answer
2k views

Weak learners for XGBoost with Tweedie distribution

Could you please explain what are the standard weak learners for XGBoost when the objective parameter equals reg:tweedie? Are they GLMs (with Tweedie distribution of dependent variable) on all ...
3
votes
2answers
1k views

Non normal residuals for Tweedie GLM

I am using Tweedie GLM as my data contains exact zeroes. However, my stats is weak and want to confirm a few things. Does Tweedie GLM assume normality of residuals? Is shapiro.test() the way for ...
1
vote
0answers
508 views

Tweedie GLM and model selection

I have run a tweedie glm since my data is zero skewed. How can I use Akaike weights and evidence ratio to choose from models the better one. Please guide. I had earlier run models with glm but with ...
1
vote
0answers
174 views

R.sq in gam model (mgcv)

first question here. I'm constructing GAMs for multispecies avian survey data, and running into some odd results in the calculation of R.sq in the summary() command. For one of my species, when i use ...
0
votes
1answer
281 views

Fitting a Poisson-Tweedie model with random effects

I'd like to fit a Poisson-Tweedie model with random effects. I'm wondering if it is possible to do it with the mcglm package. I'm following the work of Bonat et al (2017): "Extended Poisson-Tweedie: ...
2
votes
0answers
56 views

Tweedie index parameters are restricted: why?

The Tweedie distribution have variance like var(y) = $\phi \mu^p$ for any real p not between 0 and 1. I read in many places (even wikipedia) that p can take any real value except between 0 and 1. ...
5
votes
0answers
2k views

generalized linear mixed-effects models R^2 and the tweedie distribution [closed]

I am modelling data exhibiting a tweedie distribution in R using glmer (package lme4). To compare the models I would like to use the AIC and R^2. I have a couple of question on this (example code at ...
7
votes
1answer
10k views

Given a GLM using Tweedie, how do I find the coefficients?

Let $Y$ be a random variable that obeys the Tweedie distribution for parameter $\alpha = 1.1$. Let the link function be the natural log. Assume that we have a database of numbers of the form $(y_1, ...
6
votes
1answer
736 views

Skewness of Tweedie distribution

Tweedie distributions are a family of distributions from the exponential dispersion family that have power-law mean-variance relationship: \begin{align} \mathbb E[X] &= \mu \\ \operatorname{Var}[...
1
vote
0answers
1k views

deviance for zero-inflated compound poisson model, continuous data (R)

I am looking at homicide rates per US county in a given 3-yr period as a DV, against several demographic and political measures as IV. I have A LOT of zeros for the DV, and these are presumably real ...
2
votes
2answers
2k views

How do you calculate the Tweedie prediction based on model coefficients?

What is the formula to calculate the prediction for a Tweedie distribution using model coefficients? I am trying to manually calculate the prediction. Below is my attempt at reproducible code. <...
0
votes
1answer
196 views

Skewed Response Variable a lot zeroes

I am trying to build a non-parametric model to predict pure premium of an insurance policy. Pure premium is simply the expected claim amount of an insurance policy.The problem is, most insurance ...
1
vote
0answers
353 views

CDF of a Tweedie Distribution

The R package tweedie provides a way to estiamate the density and CDF of Tweedie distribution (using dtweedie and ptweedie, ...
0
votes
0answers
1k views

What's the reason for getting NaN values when predicting Tweedie GLM response?

Question is in the title. Not asking an R question, but the NaN result was in R. I just wonder why this happens for Tweedie GLMs. Example code in R, where ...
0
votes
1answer
755 views

When fitting Tweedie GLM in SAS, should I use the scale option?

In SAS proc genmod with dist=tweedie, when would you use scale=pearson or ...
31
votes
2answers
28k views

How to model non-negative zero-inflated continuous data?

I'm currently trying to apply a linear model (family = gaussian) to an indicator of biodiversity that cannot take values lower than zero, is zero-inflated and is ...
19
votes
3answers
7k views

Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?

A Tweedie distribution can model skewed data with a point mass at zero when the parameter $p$ (exponent in the mean-variance relationship) is between 1 and 2. Similarly a zero-inflated (whether ...
5
votes
1answer
1k views

A model for non-negative data with many zeros: pros and cons of Tweedie GLM

I analyze technical measurement data with the aim of developing a forecasting model. The data is given as a non-negative time series (data per hour). The data looks quite wild and contain many zeros. ...
4
votes
0answers
928 views

How a tweedie glm handles an offset?

I am trying to fit a model with a glm using a tweedie family. I use a index parameter p between 1 and 2 to get a compound Poisson Gamma distribution to fit my data. But I want to use an offset only on ...
2
votes
1answer
2k views

GLM model selection using AICc with Tweedie distribution

I have two questions regarding use of Tweedie GLM in R. I am new in using this distribution and despite a thorough search on different forums, I could not find my answers. I am now running several ...