61
votes
How to model non-negative zero-inflated continuous data?
There are a variety of solutions to the case of zero-inflated (semi-)continuous distributions:
Tobit regression: assumes that the data come from a single underlying Normal distribution, but that ...
- 41.6k
47
votes
Accepted
Is a "hurdle model" really one model? Or just two separate, sequential models?
Separating the log-likelihood
It is correct that most hurdle models can be estimated separately (I would say, instead of sequentially). The reason is that the log-likelihood can be decomposed into ...
- 15.2k
23
votes
Accepted
Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?
Note that the predicted value in a GLM is a mean.
For any distribution on non-negative values, to predict a mean of 0, its distribution would have to be entirely a spike at 0.
However, with a log-link,...
- 278k
23
votes
Accepted
Simulate from a zero-inflated poisson distribution
You can get the probability of zero-inflation by
p <- predict(object, ..., type = "zero")
and the mean of the count distribution by
...
- 15.2k
21
votes
Accepted
Why exactly can't beta regression deal with 0s and 1s in the response variable?
Because the loglikelihood contains both $\log(x)$ and $\log(1-x)$, which are unbounded when $x=0$ or $x=1$. See equation (4) of Smithson & Verkuilen, "A Better Lemon Squeezer? Maximum-Likelihood ...
- 559
19
votes
Diagnostic plots for count regression
This is an old question, but I thought it would be useful to add that my DHARMa R package (available from CRAN, see here) now provides standardized residuals for GLMs and GLMMs, based on a simulation ...
- 7,739
17
votes
Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?
Predicting the proportion of zeros
I am the author of the statmod package and joint author of the tweedie package. Everything in your example is working correctly. The code is accounting correctly ...
- 12.1k
16
votes
Zero inflated distributions, what are they really?
fit a logistic regression first calculate the probability of zeroes, and then I could remove all the zeroes, and then fit a regular regression using my choice of distribution (poisson e.g.)
You're ...
- 12.1k
15
votes
Zero inflated distributions, what are they really?
The basic idea you describe is a valid approach and it is often called a hurdle model (or two-part model) rather than a zero-inflated model.
However, it is crucial that the model for the non-zero ...
- 15.2k
15
votes
Zero Inflated Logistic Regression - Does This Exist?
Logistic regression will not "state that all future patients do not have the disease". Logistic regression yields probabilistic predictions, i.e., probabilities that a patient has the ...
- 118k
14
votes
Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?
This answer was merged from another thread asking about predictions zero-inflated regression model, but it also applies to the Tweedie GLM model.
Regression-like models predict mean of some ...
Tim♦
- 136k
13
votes
Zero Inflated Logistic Regression - Does This Exist?
While the answer by Stephan gives a good overview of the bigger picture, I think the answer in the narrow sense is IMHO that
No, zero-inflated logistic regression does not make much sense
Why? Assume ...
- 2,473
12
votes
Regression predictions show far less variance than expected
Your training data - just as any other data - is a mixture of signal and noise. In modeling, we try to capture the signal, since the noise is by definition not predictable, except in a probabilistic ...
- 118k
11
votes
Accepted
Type I and Type II negative binomial distribution in zero inflated negative binomial (ZINB) model
The difference between these two model families is the relationship between mean and variance.
nbinom1 (also called quasi-poisson)
variance = µ * phi
where µ is the mean and phi is the over-...
- 385
11
votes
Accepted
Zero inflated beta regression using gamlss for vegetation cover data
I have added preliminary support for gamlss to the emmeans package...
...
- 19.4k
11
votes
Accepted
hurdle model with non-zero gaussian distribution in R
If you want to model data that essentially follow a normal distribution for the positive values but have a point mass at zero, you could start with a Gaussian model censored at zero. In the ...
- 15.2k
10
votes
Accepted
GAMM with zero-inflated data
In addition to mgcv and its zero-inflated Poisson families (ziP() and ziplss()), you might also look at the brms package by Paul-...
- 45.8k
10
votes
Accepted
R: GLMM for unbalanced zero-inflated data (glmmTMB)
A1: "All in all, I have about 33% of the dates having counts of zero, which makes me think the data is zero inflated." -> this is a common misconception - zero-inflation != lots of zeros. Zero-...
- 7,739
10
votes
Accepted
Zero-inflated Gaussian for weights below zero recorded as 0?
I think the model is more appropriately a left-censored Gaussian, since the process you describe is about discarding information below some value (in this case, the location is known to be 0, which is ...
- 89.2k
10
votes
How do I deal with many zero values in terms of correlation?
Because you are comparing simulated vs. true values, a correlation between the two is not the best way to evaluate the quality of your simulations.
This is easy to illustrate: imagine your model is ...
- 17.7k
9
votes
Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results
I am convinced that it is incorrect to use the Vuong test -- in any of its forms -- as a test for zero-inflation. I have had a paper "The misuse of the Vuong test for non-nested models to test for ...
- 146
9
votes
Accepted
Use loess regression with many zero values
A Loess confidence interval doesn't mean much unless the Loess parameters have been cross-validated (which usually is not the case). When you use Loess for exploration, as it was originally intended, ...
- 316k
9
votes
Accepted
Is a distribution still considered right-skewed if the majority of responses are zero?
This is certainly possible. The most common definition for a distribution to be right skewed is that the skewness
$$ \gamma_1 := E\bigg(\Big(\frac{X-\mu}{\sigma}\Big)^3\bigg) $$
be positive.
For ...
- 118k
8
votes
How to test for Zero-Inflation in a dataset?
The score test (referenced in the comments by Ben Bolker) is performed by first calculating the rate estimate $\hat{\lambda}= \bar{x}$. Then count the number of observed 0s denoted $n_0$ and the total ...
- 4,189
8
votes
Zero inflated distributions, what are they really?
What ssdecontrol said is very correct. But I'd like to add a few cents to the discussion.
I just watched the lecture on Zero Inflated models for count data by Richard McElreath on YouTube.
It makes ...
- 1,468
8
votes
Accepted
Can you use glmmTMB to simultaneously model offsets and zero-inflation?
tl;dr as far as I can tell at this point,
...
- 41.6k
8
votes
Accepted
How do I deal with many zero values in terms of correlation?
Zero is a value like any other value to each kind of correlation. Each correlation takes zeros into account in its way:
as implying a deviation from the mean of either variable in the case of Pearson ...
- 54.1k
7
votes
Dealing with 0,1 values in a beta regression
I think the actual "correct" answer to this question is zero-one inflated beta regression. This is designed to handle data that vary continuously on the interval [0,1], and allows many real 0's and 1'...
- 1,182
7
votes
Fitting a model to a variable with many zeros and few but large values in right tail
If the number of counts where Count $\neq$0 is small then you can just handle this as a classification problem. Otherwise you can firstly separate the data based on the target variable into two groups:...
- 790
7
votes
Zero Inflated Logistic Regression - Does This Exist?
Zero inflated models are using a distribution (like Poisson or something else) that is mixed with a point mass at zero.
Logistic regression relates to binary data which is modelled with a Bernoulli ...
- 71.9k
Only top scored, non community-wiki answers of a minimum length are eligible