Questions tagged [tweedie-distribution]
A family of distributions from the exponential dispersion family with a power-law mean-variance relationship. For power $p$ between 1 and 2, it is a compound Poisson-Gamma distribution that has point mass at zero and is continuous on positive numbers.
72
questions
2
votes
1
answer
52
views
Distribution of a conical combination of n poisson variables?
Does a conical combination of n Poisson distributed variables have a closed-form distribution (linear combination with nonnegative coefficients)? I know that the sum of random Poisson variables would ...
- 3,070
1
vote
0
answers
49
views
How can Null model likelihood be higher than Fitted model likelihood
As far as I know, when fitting a GLM, the fitted model should always have a higher likelihood compared to the null model (with only an intercept) for the same training set. When I run a small ...
- 1,323
4
votes
1
answer
173
views
How to interpret the coefficients of Tweedie GLM with log link?
I'm trying to model cost data which have 0s. It seems that gamma is not an appropriate distribution and zero inflated gamma seems to be a bit of an overkill, but Tweedie seems to be appropriate with ...
- 71
0
votes
1
answer
35
views
obtaining a goodness of fit indices for an analysis with tweedie distribution
I performed an analysis specifying a dependent variable with a tweedie distribution using the glmmTMB package in R. I got several fit indices, specifically: AIC, BIC, Logliklihood.
What is the most ...
- 41
0
votes
0
answers
32
views
Determining p and link function for Tweedie distribution
I am trying to model mean monthly plankton values across a timeseries with a generalized additive model. My initial model was as follows:
...
- 251
0
votes
0
answers
44
views
Tweedie distribution with zeros being the second most common value
I want to perform a multilevel glm model. I have a continuous non-negative outcome variable (not count data), with many 1 values, second most common value at zero, and then a right tail of positive ...
- 41
0
votes
1
answer
108
views
Prediction of Pure Premium with offset
I'm modeling a pure premium with glm and using an offset term equal to log(exposure).
My question is, in R, what does ...
- 1
0
votes
0
answers
26
views
Modeling continuous positive data with variance structure similar to Tweedie with 1.2 variance power, but without excess zeros
I'm working on a project where I need to model continuous positive data. I found that a Tweedie distribution with a variance power around 1.2 and a dispersion parameter just above 1 fits my data quite ...
- 11
1
vote
0
answers
49
views
Appropriateness of Tweedie GLM for modeling average daily driving distance with unknown numerator and denominator
I have a dataset of cars and want to model a variable called average daily driving distance. The variable was calculated before I received the dataset as:
average daily driving distance = total ...
- 11
2
votes
1
answer
185
views
Use of weights in choosing power parameter in Tweedie distribution
I'm looking at the implementation of the tweedie.profile function from the R package tweedie. I have a few questions.
When ...
- 1,323
0
votes
0
answers
150
views
Creating a Generalized Additive Model for Zero-Inflated Proportion Response Variable
I am a PhD Candidate just beginning the data analysis phase of my dissertation. I have minimal experience with statistical modeling, so I have had to do a lot of self-teaching so please bear with me.
...
0
votes
0
answers
75
views
Loss Function for Neural Network with High density at 0
I am working on a time series project and looking to use a transformer based Neural Network (specifically, temporal fusion transformer).
My data is extremely heavily at 0 (the use case is that most ...
- 21
7
votes
1
answer
890
views
Tweedie Dispersion Parameter Estimation Methods
In the book Generalized Linear Models with Examples in R - Dunn and Smyth, in Chapter 6.8, it is recommended to use the Pearson estimator of the dispersion - "This makes the Pearson estimator the ...
- 1,323
1
vote
1
answer
34
views
Prudent to reduce data size for the sake of model performance?
I am currently working on predicting the customer revenue in next 3,6 or 9 months using the below two methods
a) Buy Till you die probabilistic models
b) Tweedie regression and other regression ...
- 2,936
1
vote
0
answers
125
views
Dealing with lower- and upper-bound truncated data with Tweedie distribution through glmmTMB
I'm working with intensive longitudinal data with observations nested within persons via glmmTMB with a random effect of person. My outcome is negative emotion, which was computed by taking the mean ...
- 11
4
votes
2
answers
279
views
GAM: Find a good distribution for the monthly data sums?
I am new in the GAM modelling. I would like to find a family, that will fit my response variables. I am using the sums of monthly counts of beetles, collected from the beetle traps in ~ two weeks ...
- 273
7
votes
1
answer
1k
views
When should one use a Tweedie GLM over a Zero-Inflated GLM?
I have seen both Tweedie GLMs and Zero-Inflated (ZI) GLMs used in the field of ecology. Tweedie seems to have the benefit of not treating excess zeros separately, as is done using ZI regressions. ...
- 409
1
vote
1
answer
300
views
Incorporate Weights/Offsets with Nonparametric Models
I am modeling pure premium in R. I have read that pure premiums are usually modeled using a Tweedie distribution (glm). There is generally an offset or weight added to the model, such as an exposure. ...
- 265
6
votes
1
answer
2k
views
Understanding the Tweedie Distribution
I am trying to better understand the basic details and properties about the Tweedie Probability Distribution.
The Tweedie Probability Distribution is a bimodal probability distribution function which ...
1
vote
0
answers
291
views
why is gradient boosting machine based algorithm nonparametric although they have a loss function?
I am dealing with very right-skewed distribution of y which seems to follow a tweedie distribution. and I found out it would give a higher performance to change lgbm's loss function to a "tweedie&...
- 11
3
votes
1
answer
679
views
Interpreting tweedie-location-scale result from R mgcv package
I came across the Tweedie-location-scale family function in R (twlss function from mgcv). It looks appealing to model my data. I understand the general concepts is that "twlss" family allows ...
- 133
0
votes
0
answers
249
views
Tweedie distributions for large dataset, power parameter for different density peaks and random effects
I have a large dataset (343750 variables and 151 observations) and I want to model each variable as a response one in order to know if it can be explained by the group of patients and the gene ...
2
votes
2
answers
1k
views
Tweedie GLM, what var.power should I chose?
Im trying to do a GLM on my data about bats to see how different variables affect bat activity on 8 species. Orignally my data was count data, but because of hardware difficulties I had to divide the ...
1
vote
0
answers
447
views
Comparing the fit of Tweedie and gamma hurdle models
I am comparing several modeling approaches to semi-continuous data (many exact zeros and continuous positive cost outcomes) to assess the effect of the main predictor "disease" on cost.
The ...
- 211
2
votes
1
answer
2k
views
Python statsmodels constant and error using Tweedie distribution
I'm following a paper described in the link which models rainfall. Due to a cyclical nature the authors use a model equation (53):
$$
log(\mu_{i}) = a_{0} + a_{1} sin (\frac{2\pi i}{365}) + a_{2} ...
- 123
17
votes
1
answer
8k
views
What is use of Tweedie or poisson loss/objective function in XGboost and Deep learning models
I am looking at few competitions in kaggle where people used tweedie loss or poisson loss as objective function for forecasting sales or predicting insurance claims.
Can someone please explain the ...
- 797
4
votes
0
answers
2k
views
How do I choose the correct power in a tweedie GLM?
I'm developing a GLM with a tweedie family distribution. How do I choose P correctly and how much does it matter?
I know it has to be between 1 and 2, and when I try different values, I don't see an ...
- 235
2
votes
0
answers
990
views
XGBoost/LightGBM underpredicts for some categories but is otherwise good
I am trying to predict the revenue that a click generates on an average (revenue per click). There are K different items that get shown to different users. My training data consist of clicks. A ...
- 1,389
1
vote
0
answers
189
views
Comparing a hurdle model with a "direct" model
I am building xgboost models for prediction of insurance risk, the risk being assumed to follow a tweedie distribution with tweedie variance power between 1 and 2 (https://en.wikipedia.org/wiki/...
- 75
0
votes
1
answer
528
views
How do I manually predict Tweedie GLM
I have a simple Tweedie model in GLM, or $Y_i=\beta_0+\beta_1x_1$. As usual, $Y$ is the output and $x_i$ is its dependent variable. Suppose that $p>1$, my link function would be $\frac{\mu^{1-p}}{1-...
- 111
1
vote
0
answers
69
views
R: tweedie.dev function gives Inf
I have a question regarding the function tweedie.dev(y, mu, power) from the tweedie package. When I run this function on some data I have generated I get Inf where y = 0 (I use power = 1.5). Does this ...
- 75
0
votes
1
answer
199
views
Identifying the best distribution to this data?
I'm trying to fit an appropriate distribution to a data with 216 values and estimate parameters. From Cullen and Frey graph, it looks like lognormal could be a good fit. From q-q plot, Weibull seems ...
- 125
1
vote
0
answers
159
views
Generate random variates and cdf from Tweedie distribution
I have to generate some random variates from a Tweedie distribution with the following parameters:
$p = 1$
$\mu = \bar{\mu}$
$\phi = \bar{\phi}$
I do not have direct access to a function that is able ...
3
votes
1
answer
1k
views
How to estimate power index for tweedie glm on large data
I've went through the tweedie chapter in the book Generalized Linear Models with Examples in R and it seems that the function tweedie.profile is used to estimate the power index for tweedie glm.
...
- 41
2
votes
1
answer
192
views
What does it mean when MAE improved while RMSE worsened?
I am comparing two models:
One is a black box that I cannot understand, the other is a GLM.
How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
- 135
4
votes
1
answer
2k
views
GLM Tweedie dispersion parameter
I am running a tweedie regression and I need to get the dispersion parameter(phi).
Here a simple R code:
...
3
votes
1
answer
1k
views
Overdispersion vs Tweedie
I am dealing with data that could possibly be overdispersed and I am looking at fitting a GLM with a quasi distribution.
As far as I understand, when we fit a glm ...
2
votes
0
answers
106
views
Replicating a Tweedie corrected experiment from Computer Age Statistical Inference
I have been attempting to replicate an experiment from Computer Age Statistical Inference by Bradley Efron and Trevor Hastie on page 411.
In this experiment 100 datasets are populated normal random ...
- 1,921
2
votes
1
answer
2k
views
Tweedie distribution without zeroes
Recently I've found the Tweedie distribution useful for modelling my plant shoot weight data in glmmTMB. I started using it because my shoot weight datasets have many zeroes, and it has yielded better ...
- 63
7
votes
1
answer
4k
views
Chi Square vs F Tests for GLM Model Comparisons
I've been creating some models in R using glm() and rxGlm(). I'm experienced in building GLMs but my memory of some of the ...
- 211
0
votes
0
answers
122
views
How do I properly choose a value for 'phi'
How does one choose the proper value for the phi parameter in a random tweedie sample.
I'm trying to properly choose the correct value for the 'phi' parameter for ...
- 235
4
votes
2
answers
2k
views
Does the dependent variable in a GLM have to be transformed before running the model or does the model do it?
I'm trying to fit a Tweedie model with statsmodels and was wondering if I have to transform the dependent variable before I run the model or if ...
- 235
2
votes
0
answers
275
views
SAS and R gives different standard error when fitting Tweedie model
I would like to fit a Tweedie GLM to the data. However, even though PROC GENMOD from SAS and glm() from R gives the same coefficient estimates, they give quite different standard error. What it the ...
- 1,676
0
votes
1
answer
1k
views
h2o glm tweedie for categorical variables?
To build a tweedie glm for categorical variables, the document suggested that I can use data['variable_name'].asfactor().
However, in the model output, there is ...
3
votes
0
answers
237
views
Best way to test the difference in means for Tweedie distributions?
What is the best way to test the difference in means for two samples of data that follow a Tweedie distribution?
(zero inflated, tweedie power parameter ~1.7)
The data represents the number of ...
- 143
3
votes
1
answer
3k
views
Weak learners for XGBoost with Tweedie distribution
Could you please explain what are the standard weak learners for XGBoost
when the objective parameter equals reg:tweedie?
Are they GLMs (with Tweedie distribution of dependent variable) on
all ...
- 221
4
votes
2
answers
2k
views
Non normal residuals for Tweedie GLM
I am using Tweedie GLM as my data contains exact zeroes. However, my stats is weak and want to confirm a few things.
Does Tweedie GLM assume normality of residuals?
Is shapiro.test() the way for ...
- 81
1
vote
0
answers
644
views
Tweedie GLM and model selection
I have run a tweedie glm since my data is zero skewed. How can I use Akaike weights and evidence ratio to choose from models the better one. Please guide. I had earlier run models with glm but with ...
- 81
1
vote
0
answers
201
views
R.sq in gam model (mgcv)
first question here. I'm constructing GAMs for multispecies avian survey data, and running into some odd results in the calculation of R.sq in the summary() command. For one of my species, when i use ...
- 33
0
votes
1
answer
393
views
Fitting a Poisson-Tweedie model with random effects
I'd like to fit a Poisson-Tweedie model with random effects. I'm wondering if it is possible to do it with the mcglm package. I'm following the work of Bonat et al (2017): "Extended Poisson-Tweedie: ...
- 3