Skip to main content

Questions tagged [tweedie-distribution]

A family of distributions from the exponential dispersion family with a power-law mean-variance relationship. For power $p$ between 1 and 2, it is a compound Poisson-Gamma distribution that has point mass at zero and is continuous on positive numbers.

Filter by
Sorted by
Tagged with
1 vote
1 answer
19 views

RMSE on Individual vs. Aggregate Data

I am comparing two scenarios: A model is made and predictions obtained on a test set. Then deciles are formed by ordering the prediction and making 10 buckets of equal volume according to a weight. ...
user2997345's user avatar
0 votes
0 answers
11 views

Variance decomposition with Tweedie distribution – back transform necessary?

We are running a hierarchical random intercept model with a Tweedie distributed dependent variable (see below) and three levels of hierarchy. Our aim is to estimate how much of the total variance (in ...
james_westfield's user avatar
4 votes
1 answer
54 views

Can GAM with Tweedie distribution be used for a binary response?

I am trying species distribution models with GAM. The data set is the presence/absence of a species and environment variables in each grid. In the paper of Virgili et al. (2018), the authors suggested ...
Terpsiphone's user avatar
0 votes
1 answer
61 views

Why does Tweedie Deviance not contain the dispersion parameter?

I was looking at the tweedie deviance function, and I noticed it only seems to contain the parameters for the mean and the power parameter. But intuitively, I would think the dispersion parameter ...
Ty4Reading's user avatar
2 votes
1 answer
63 views

Distribution of a conical combination of n poisson variables?

Does a conical combination of n Poisson distributed variables have a closed-form distribution (linear combination with nonnegative coefficients)? I know that the sum of random Poisson variables would ...
Tom Wenseleers's user avatar
1 vote
1 answer
78 views

How can Null model likelihood be higher than Fitted model likelihood

As far as I know, when fitting a GLM, the fitted model should always have a higher likelihood compared to the null model (with only an intercept) for the same training set. When I run a small ...
Kozolovska's user avatar
  • 1,355
4 votes
1 answer
500 views

How to interpret the coefficients of Tweedie GLM with log link?

I'm trying to model cost data which have 0s. It seems that gamma is not an appropriate distribution and zero inflated gamma seems to be a bit of an overkill, but Tweedie seems to be appropriate with ...
user395714's user avatar
0 votes
1 answer
93 views

obtaining a goodness of fit indices for an analysis with tweedie distribution

I performed an analysis specifying a dependent variable with a tweedie distribution using the glmmTMB package in R. I got several fit indices, specifically: AIC, BIC, Logliklihood. What is the most ...
Lior's user avatar
  • 41
0 votes
0 answers
155 views

Determining p and link function for Tweedie distribution

I am trying to model mean monthly plankton values across a timeseries with a generalized additive model. My initial model was as follows: ...
tnt's user avatar
  • 283
0 votes
0 answers
80 views

Tweedie distribution with zeros being the second most common value

I want to perform a multilevel glm model. I have a continuous non-negative outcome variable (not count data), with many 1 values, second most common value at zero, and then a right tail of positive ...
Lior's user avatar
  • 41
0 votes
1 answer
195 views

Prediction of Pure Premium with offset

I'm modeling a pure premium with glm and using an offset term equal to log(exposure). My question is, in R, what does ...
Abdou's user avatar
  • 1
1 vote
0 answers
57 views

Appropriateness of Tweedie GLM for modeling average daily driving distance with unknown numerator and denominator

I have a dataset of cars and want to model a variable called average daily driving distance. The variable was calculated before I received the dataset as: average daily driving distance = total ...
Chebyshev's user avatar
2 votes
1 answer
269 views

Use of weights in choosing power parameter in Tweedie distribution

I'm looking at the implementation of the tweedie.profile function from the R package tweedie. I have a few questions. When ...
Kozolovska's user avatar
  • 1,355
0 votes
0 answers
280 views

Creating a Generalized Additive Model for Zero-Inflated Proportion Response Variable

I am a PhD Candidate just beginning the data analysis phase of my dissertation. I have minimal experience with statistical modeling, so I have had to do a lot of self-teaching so please bear with me. ...
Adrian Figueroa's user avatar
7 votes
1 answer
1k views

Tweedie Dispersion Parameter Estimation Methods

In the book Generalized Linear Models with Examples in R - Dunn and Smyth, in Chapter 6.8, it is recommended to use the Pearson estimator of the dispersion - "This makes the Pearson estimator the ...
Kozolovska's user avatar
  • 1,355
1 vote
1 answer
36 views

Prudent to reduce data size for the sake of model performance?

I am currently working on predicting the customer revenue in next 3,6 or 9 months using the below two methods a) Buy Till you die probabilistic models b) Tweedie regression and other regression ...
The Great's user avatar
  • 3,302
1 vote
0 answers
163 views

Dealing with lower- and upper-bound truncated data with Tweedie distribution through glmmTMB

I'm working with intensive longitudinal data with observations nested within persons via glmmTMB with a random effect of person. My outcome is negative emotion, which was computed by taking the mean ...
kmpmckone's user avatar
5 votes
2 answers
335 views

GAM: Find a good distribution for the monthly data sums?

I am new in the GAM modelling. I would like to find a family, that will fit my response variables. I am using the sums of monthly counts of beetles, collected from the beetle traps in ~ two weeks ...
maycca's user avatar
  • 355
11 votes
1 answer
2k views

When should one use a Tweedie GLM over a Zero-Inflated GLM?

I have seen both Tweedie GLMs and Zero-Inflated (ZI) GLMs used in the field of ecology. Tweedie seems to have the benefit of not treating excess zeros separately, as is done using ZI regressions. ...
geoscience123's user avatar
1 vote
1 answer
405 views

Incorporate Weights/Offsets with Nonparametric Models

I am modeling pure premium in R. I have read that pure premiums are usually modeled using a Tweedie distribution (glm). There is generally an offset or weight added to the model, such as an exposure. ...
mapleleaf's user avatar
  • 275
10 votes
1 answer
5k views

Understanding the Tweedie Distribution

I am trying to better understand the basic details and properties about the Tweedie Probability Distribution. The Tweedie Probability Distribution is a bimodal probability distribution function which ...
stats_noob's user avatar
1 vote
0 answers
329 views

why is gradient boosting machine based algorithm nonparametric although they have a loss function?

I am dealing with very right-skewed distribution of y which seems to follow a tweedie distribution. and I found out it would give a higher performance to change lgbm's loss function to a "tweedie&...
Kosney's user avatar
  • 11
3 votes
1 answer
837 views

Interpreting tweedie-location-scale result from R mgcv package

I came across the Tweedie-location-scale family function in R (twlss function from mgcv). It looks appealing to model my data. I understand the general concepts is that "twlss" family allows ...
xing_yu's user avatar
  • 133
2 votes
0 answers
511 views

Can weights be used in place of offset in Tweedie model

I have seen evidence that Poisson regression can be run with either offsets or weights for time and both give the same results. However, I have noticed that the same is not true for Tweedie. I have ...
magooken's user avatar
0 votes
0 answers
286 views

Tweedie distributions for large dataset, power parameter for different density peaks and random effects

I have a large dataset (343750 variables and 151 observations) and I want to model each variable as a response one in order to know if it can be explained by the group of patients and the gene ...
Elvira Nightingale's user avatar
2 votes
2 answers
2k views

Tweedie GLM, what var.power should I chose?

Im trying to do a GLM on my data about bats to see how different variables affect bat activity on 8 species. Orignally my data was count data, but because of hardware difficulties I had to divide the ...
Peter Sivertsen's user avatar
1 vote
0 answers
576 views

Comparing the fit of Tweedie and gamma hurdle models

I am comparing several modeling approaches to semi-continuous data (many exact zeros and continuous positive cost outcomes) to assess the effect of the main predictor "disease" on cost. The ...
Kellan Baker's user avatar
2 votes
1 answer
2k views

Python statsmodels constant and error using Tweedie distribution

I'm following a paper described in the link which models rainfall. Due to a cyclical nature the authors use a model equation (53): $$ log(\mu_{i}) = a_{0} + a_{1} sin (\frac{2\pi i}{365}) + a_{2} ...
sdbol's user avatar
  • 123
20 votes
1 answer
10k views

What is use of Tweedie or poisson loss/objective function in XGboost and Deep learning models

I am looking at few competitions in kaggle where people used tweedie loss or poisson loss as objective function for forecasting sales or predicting insurance claims. Can someone please explain the ...
tjt's user avatar
  • 827
4 votes
0 answers
2k views

How do I choose the correct power in a tweedie GLM?

I'm developing a GLM with a tweedie family distribution. How do I choose P correctly and how much does it matter? I know it has to be between 1 and 2, and when I try different values, I don't see an ...
Jordan 's user avatar
  • 235
2 votes
0 answers
1k views

XGBoost/LightGBM underpredicts for some categories but is otherwise good

I am trying to predict the revenue that a click generates on an average (revenue per click). There are K different items that get shown to different users. My training data consist of clicks. A ...
Nik's user avatar
  • 1,389
1 vote
0 answers
213 views

Comparing a hurdle model with a "direct" model

I am building xgboost models for prediction of insurance risk, the risk being assumed to follow a tweedie distribution with tweedie variance power between 1 and 2 (https://en.wikipedia.org/wiki/...
Unknown's user avatar
  • 75
0 votes
1 answer
613 views

How do I manually predict Tweedie GLM

I have a simple Tweedie model in GLM, or $Y_i=\beta_0+\beta_1x_1$. As usual, $Y$ is the output and $x_i$ is its dependent variable. Suppose that $p>1$, my link function would be $\frac{\mu^{1-p}}{1-...
tkhu's user avatar
  • 111
1 vote
0 answers
75 views

R: tweedie.dev function gives Inf

I have a question regarding the function tweedie.dev(y, mu, power) from the tweedie package. When I run this function on some data I have generated I get Inf where y = 0 (I use power = 1.5). Does this ...
Unknown's user avatar
  • 75
0 votes
1 answer
225 views

Identifying the best distribution to this data?

I'm trying to fit an appropriate distribution to a data with 216 values and estimate parameters. From Cullen and Frey graph, it looks like lognormal could be a good fit. From q-q plot, Weibull seems ...
Mohamad Sahil's user avatar
1 vote
0 answers
198 views

Generate random variates and cdf from Tweedie distribution

I have to generate some random variates from a Tweedie distribution with the following parameters: $p = 1$ $\mu = \bar{\mu}$ $\phi = \bar{\phi}$ I do not have direct access to a function that is able ...
Marco De Virgilis's user avatar
3 votes
1 answer
1k views

How to estimate power index for tweedie glm on large data

I've went through the tweedie chapter in the book Generalized Linear Models with Examples in R and it seems that the function tweedie.profile is used to estimate the power index for tweedie glm. ...
RamenZzz's user avatar
2 votes
1 answer
217 views

What does it mean when MAE improved while RMSE worsened?

I am comparing two models: One is a black box that I cannot understand, the other is a GLM. How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
Seraphim's user avatar
  • 135
5 votes
1 answer
2k views

GLM Tweedie dispersion parameter

I am running a tweedie regression and I need to get the dispersion parameter(phi). Here a simple R code: ...
Marco De Virgilis's user avatar
3 votes
1 answer
1k views

Overdispersion vs Tweedie

I am dealing with data that could possibly be overdispersed and I am looking at fitting a GLM with a quasi distribution. As far as I understand, when we fit a glm ...
Marco De Virgilis's user avatar
2 votes
0 answers
108 views

Replicating a Tweedie corrected experiment from Computer Age Statistical Inference

I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411. In this experiment 100 datasets are populated normal random ...
Alex's user avatar
  • 2,021
3 votes
1 answer
2k views

Tweedie distribution without zeroes

Recently I've found the Tweedie distribution useful for modelling my plant shoot weight data in glmmTMB. I started using it because my shoot weight datasets have many zeroes, and it has yielded better ...
Niall Millar's user avatar
7 votes
1 answer
5k views

Chi Square vs F Tests for GLM Model Comparisons

I've been creating some models in R using glm() and rxGlm(). I'm experienced in building GLMs but my memory of some of the ...
Alan's user avatar
  • 211
0 votes
0 answers
132 views

How do I properly choose a value for 'phi'

How does one choose the proper value for the phi parameter in a random tweedie sample. I'm trying to properly choose the correct value for the 'phi' parameter for ...
Jordan 's user avatar
  • 235
4 votes
2 answers
2k views

Does the dependent variable in a GLM have to be transformed before running the model or does the model do it?

I'm trying to fit a Tweedie model with statsmodels and was wondering if I have to transform the dependent variable before I run the model or if ...
Jordan 's user avatar
  • 235
2 votes
0 answers
290 views

SAS and R gives different standard error when fitting Tweedie model

I would like to fit a Tweedie GLM to the data. However, even though PROC GENMOD from SAS and glm() from R gives the same coefficient estimates, they give quite different standard error. What it the ...
3x89g2's user avatar
  • 1,716
0 votes
1 answer
1k views

h2o glm tweedie for categorical variables?

To build a tweedie glm for categorical variables, the document suggested that I can use data['variable_name'].asfactor(). However, in the model output, there is ...
Jie Huang's user avatar
3 votes
0 answers
306 views

Best way to test the difference in means for Tweedie distributions?

What is the best way to test the difference in means for two samples of data that follow a Tweedie distribution? (zero inflated, tweedie power parameter ~1.7) The data represents the number of ...
Harry M's user avatar
  • 143
3 votes
1 answer
3k views

Weak learners for XGBoost with Tweedie distribution

Could you please explain what are the standard weak learners for XGBoost when the objective parameter equals reg:tweedie? Are they GLMs (with Tweedie distribution of dependent variable) on all ...
Ievgen's user avatar
  • 231
4 votes
2 answers
2k views

Non normal residuals for Tweedie GLM

I am using Tweedie GLM as my data contains exact zeroes. However, my stats is weak and want to confirm a few things. Does Tweedie GLM assume normality of residuals? Is shapiro.test() the way for ...
Harshad's user avatar
  • 81