Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

Filter by
Sorted by
Tagged with
88
votes
4answers
35k views

Diagnostic plots for count regression

What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable? I'm especially interested in Poisson and negative binomial models, ...
81
votes
4answers
43k views

What is the difference between zero-inflated and hurdle models?

I wonder if there is a clear-cut difference between the so-called zero-inflated distributions (models) and so-called hurdle-at-zero distributions (models)? The terms occur quite often in the ...
25
votes
3answers
4k views

Is a “hurdle model” really one model? Or just two separate, sequential models?

Consider a hurdle model predicting count data y from a normal predictor x: ...
21
votes
1answer
8k views

When to use Poisson vs. geometric vs. negative binomial GLMs for count data?

I'm trying to layout for myself when it's appropriate to use which regression type (geometric, Poisson, negative binomial) with count data, within the GLM framework (only 3 of the 8 GLM distributions ...
20
votes
4answers
5k views

Dealing with 0,1 values in a beta regression

I have some data in [0,1] which I would like to analyze with a beta regression. Of course something needs to be done to accommodate the 0,1 values. I dislike modifying data to fit a model. also I ...
18
votes
5answers
9k views

Beta regression of proportion data including 1 and 0

I am trying to produce a model for which I have a response variable which is a proportion between 0 and 1, this includes quite a few 0s and 1s but also many values in between. I am thinking about ...
17
votes
2answers
4k views

Why exactly can't beta regression deal with 0s and 1s in the response variable?

Beta regression (i.e. GLM with beta distribution and usually the logit link function) is often recommended to deal with response aka dependent variable taking values between 0 and 1, such as fractions,...
17
votes
3answers
13k views

Zero-inflated negative binomial mixed-effects model in R

Is there such a package that provides for zero-inflated negative binomial mixed-effects model estimation in R? By that I mean: Zero-inflation where you can specify the binomial model for zero ...
16
votes
0answers
573 views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
15
votes
1answer
14k views

How to model non-negative zero-inflated continuous data?

I'm currently trying to apply a linear model (family = gaussian) to an indicator of biodiversity that cannot take values lower than zero, is zero-inflated and is ...
14
votes
3answers
4k views

Can a model for non-negative data with clumping at zeros (Tweedie GLM, zero-inflated GLM, etc.) predict exact zeros?

A Tweedie distribution can model skewed data with a point mass at zero when the parameter $p$ (exponent in the mean-variance relationship) is between 1 and 2. Similarly a zero-inflated (whether ...
14
votes
1answer
869 views

Zero-inflated Poisson regression

Suppose $ \textbf{Y} = (Y_1, \dots, Y_n)'$ are independent and $$\eqalign{ Y_i = 0 & \text{with probability} \ p_i+(1-p_i)e^{-\lambda_i}\\ Y_i = k & \text{with probability} \ (1-p_i)e^{-\...
12
votes
3answers
5k views

Zero inflated distributions, what are they really?

I am struggling to understand zero inflated distributions. What are they? What's the point? If I have data with many zeroes, then I could fit a logistic regression first calculate the probability of ...
12
votes
1answer
5k views

Trouble finding good model fit for count data with mixed effects - ZINB or something else?

I have a very small data set on solitary bee abundance that I am having trouble analysing. It’s count data, and almost all the counts are in one treatment with most of the zeroes in the other ...
11
votes
2answers
12k views

Zero-inflated count models in R: what is the real advantage?

For analysing zero-inflated bird counts I'd like to apply zero-inflated count models using the R package pscl. However, having a look at the example provided in the documentation for one of the main ...
11
votes
1answer
11k views

Mean and variance of a zero-inflated Poisson distribution

Can anyone show how the expected value and variance of the zero inflated Poisson, with probability mass function $$ f(y) = \begin{cases} \pi+(1-\pi)e^{-\lambda}, & \text{if }y=0 \\ (1-\pi)\frac{\...
11
votes
3answers
2k views

GLM with continuous data piled up at zero

I am trying to run a model to estimate how well catastrophic illnesses such as TB, AIDS etc affect spending on hospitalization. I have "per hospitalization cost" as the dependent variable and various ...
11
votes
2answers
16k views

Poisson regression assumptions and how to test them in R

I would like to test in what regression fits my data best. My dependent variable is a count, and has a lot of zeros. And I would need some help to determine what model and family to use (poisson or ...
11
votes
2answers
7k views

Proper use and interpretation of zero-inflated gamma models

Background: I am a biostatistician presently wrestling with a dataset of cellular expression rates. The study exposed a host of cells, collected in groups from various donors, to certain peptides. ...
11
votes
2answers
4k views

How can I set up a zero-inflated poisson in JAGS?

I am trying to set up a zero-inflated poisson model in R and JAGS. I am new to JAGS and I need some guidance on how to do that. I've been trying with the following where y[i] is the observed ...
11
votes
1answer
869 views

Measure of “deviance” for zero-inflated Poisson or zero-inflated negative binomial?

Scaled deviance, defined as D = 2 * (log-likelihood of saturated model minus log-likelihood of fitted model), is often used as a measure of goodness-of-fit in GLM models. Percent deviance explained, ...
11
votes
0answers
2k views

Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...
9
votes
2answers
1k views

GAMM with zero-inflated data

Is it possible to fit a GAMM(Generalized Additive Mixed Model) for zero-inflated data in R? If not, is it possible to fit a GAM(Generalized Additive Model) for zero-inflated data with a negative ...
9
votes
2answers
633 views

Are a zero-truncated Poisson and basic Poisson nested or non-nested?

I've seen plenty that discusses whether a basic Poisson regression is a nested version of a zero-inflated Poisson regression. For instance this site argues that it is, since the latter includes extra ...
9
votes
3answers
3k views

How to test/prove data is zero inflated?

I've got a problem that I think should be simple but can't quite figure it out. I'm looking at seed pollination, I have plants (n=36) that flower in clusters, I sample 3 flower clusters from each ...
9
votes
1answer
4k views

How to get standard errors from R zero-inflated count data regression? [closed]

The following code PredictNew <- predict (glm.fit, newdata = Predict, X1 =X1, Y1= Y1, type = "response", se.fit = TRUE) produces a 3-...
8
votes
2answers
1k views

Why are there no one-inflated count data models?

I am working on zero-inflated count data models using the pscl package. I am just wondering why there is no development of models for one-inflated count data models!...
7
votes
1answer
4k views

Simulate from a zero-inflated poisson distribution

I am trying to simulate from observed data that I have fit to a zero-inflated poisson regression model. I fit the data in R using zeroinfl() from the package pscl, but I am having trouble figuring out ...
7
votes
3answers
3k views

Zero inflated models - “true zero” vs. “excess zero”

I am trying to decide if zero inflated poisson is appropriate for my data vs. a Poisson hurdle model. In background reading between the two I've run across a statement saying that a zero inflated ...
7
votes
3answers
5k views

How to test for Zero-Inflation in a dataset?

I have a dataset which seems to have a lot of zeroes. I have already fit a poisson regression model as well as a negative binomial model. I would like to fit zero-inflated and hurdle models as well. ...
7
votes
2answers
250 views

Experimental design & questions on use of generalized linear models

I have an ecological experiment for which I need to analyze bird count data. Here is the set up: 2 treatments (open/control), 3 regions. Not quite a full 3x2 factorial because in 2 regions there are ...
7
votes
1answer
474 views

Zero-inflated negative binomial models: why not use two separate models?

Zero-inflated negative binomial models have two components: a count component (negative binomial regression part) and a zero component (logistic regression part). Why not just run two separate ...
7
votes
2answers
2k views

Zero-inflated two-part models for semi-continuous data

I am trying to study predictors of companies' pollution output of some specific chemicals. The data I am using have many 0's (i.e., the company did not pollute at all with those chemicals) and then ...
7
votes
1answer
2k views

Exact difference between two-part models (e.g., Cragg) and Tobit type 2 models (e.g., Heckman)

I want to run a regression where the DV is the amount of funding (in USD) obtained by startups. Naturally the DV contains a lot of zero's (~55%) and has a continuous distribution for y>0. In general ...
7
votes
0answers
5k views

“Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...
6
votes
3answers
5k views

Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results

I'm analyzing count data for a set of ten species and found that for the five species with highest detection rate, the zero-inflated poisson (ZIP) regression fits the data significantly better than ...
6
votes
1answer
556 views

Use loess regression with many zero values

I have measuments of vegetation coverage on Y plotted against surface height (and hence flooding frequency) on X. The vegetation often has two herb layers, which are estimated seperately. If only one ...
6
votes
1answer
5k views

Interpretation of $\theta$ in negative binomial regression

First off, a very similar question has been asked before. But the answers to this question did not explain what high/low values of theta mean. Here's my crack at trying to figure out what high/low ...
6
votes
1answer
5k views

Interpret zero-inflated negative binomial regression

I am trying to estimate a zero-inflated negative binomial model with 11 predictor variables and the number of reported crimes as a response variable. The model seems to work OK, but I'm uncertain on ...
6
votes
1answer
3k views

Please help me refine this zero-inflated negative binomial model

I have been working on a baseball model to predict success at the major league level using minor league statistics. After posting multiple threads on this site (1, 2, 3) and receiving valuable ...
6
votes
1answer
2k views

Zero-inflation on steroids: choose among Poisson, negative binomial and zero-inflated regressions

I am struggling to fit alternative count models into my data. I guess my problem is just too many zeros. This is my data ...
5
votes
3answers
3k views

Fitting a probability distribution to zero inflated data in R

I am trying to learn how to fit a probability distribution to a vector of data, using the program R, but there are a lot of potential probability distributions to use! So my question is, how do I ...
5
votes
1answer
8k views

Fitting a model to a variable with many zeros and few but large values in right tail [duplicate]

I would like to fit a model to a dependent variable distributed like the one below (see picture). The distribution is a count of people (with specific characteristics) in various districts. This ...
5
votes
1answer
2k views

Enormous SEs in zero-inflated negative binomial regression

I have overdispersed count data where the outcome is events (occurrence of a rare disease) and the covariate of interest is season. The unit of analysis is the number of events occurring in a country-...
5
votes
2answers
6k views

Modelling zero-inflated proportion data in R using GAMLSS

I am new to the gamlss package and would like to check that I am using the correct family for proportion data (tree species cover after treatment), which is bounded ...
5
votes
1answer
2k views

Explanatory variables with many zeros

I am trying to fit a linear model to a price response variable. Many of the predictor variables consist of mainly zeros. For example, one possible predictor variable is "drill holes". Not many parts ...
5
votes
1answer
3k views

Zero inflated beta regression using gamlss for vegetation cover data

My goal is to analyse vegetation cover data. The way the data collection works is that you throw a quadrat (0.5m x 0.5m) in a sample plot and estimate the percent cover of the target species. Here is ...
5
votes
1answer
10k views

When to use zero-inflated poisson regression and negative binomial distribution

I have a fairly simple dataset looking at the relationship between the first nesting date of a bird in a given year (Date) and the birds overall fledgling production from that year (Fledge; count data ...
5
votes
1answer
621 views

Dealing with imbalanced/zero-inflated training examples for regression

I am trying to predict the rainfall in a desert with a regression model. However, as you might expect, most of my training examples have zeroed labels. I have two questions: a. What is an appropriate ...
5
votes
1answer
899 views

Zero inflated negative binomial with selection

I am looking for a Stata (or R/Matlab if there's no Stata) implementation of the model described by Greene (1994) (http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1293115). It is essentially a ...