The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

157 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
16
votes
0answers
574 views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
11
votes
0answers
2k views

Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...
7
votes
0answers
5k views

“Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...
5
votes
0answers
618 views

Does there exist zero-inflated linear regression?

I have a non-count data with huge number of zeros in the target variable. I need to fit a model being a mixture of Dirac delta function and normal distribution parametrized by mean $X\beta$ and ...
5
votes
0answers
626 views

Gibbs sampling deriving complete conditionals with mixture priors

My question is about the derivation of the complete conditionals for Gibbs sampling in a hierarchical model where some of the parameters are mixtures of point-masses and Normal distributions. The ...
4
votes
0answers
3k views

Zero-inflated model: non-finite value supplied by optim

So I have the following model predicting the presence of an animal on a certain spot. As a time unit quarter is initially used, but for one of the species of animals there is some (little) interesting ...
4
votes
0answers
458 views

How to write unnormalized posterior when prior is a mixture of continuous and discrete

Suppose I want to do bayesian inference on the regression problem $\beta$ for Y = X$\beta$ + $\epsilon$ for $\epsilon_i \sim N(0,\sigma^2)$. The complication is that the prior for each component $\...
4
votes
0answers
461 views

Basic idea of zero inflated two part models(hurdel) and application to big data (machine learning)

I'm currently working on the data which has 90% 0s in response variable. Based on my research, it seems zero inflated models could be a solution to this. However, while I was reading related documents,...
4
votes
0answers
9k views

Fitting a zero-inflated negative binomial regression with R

In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ...
3
votes
3answers
198 views

Compare two samples with many zeros

We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ...
3
votes
1answer
58 views

Contradiction between zero-inflated poisson model coefficients and graph of the model?

EDIT: Added an reproducible example For one of my models, it seems the coefficients and the graphed out model do not agree. I'm working with adverse effects data, in which intense reactions are rare ...
3
votes
0answers
456 views

Negative binomial model vs zero-inflated negative binomial - theoretical justifications

I have a count variable that I would like to predict using a categorical variable (it has 4 levels). I would like to decide whether I should use Poisson, negative binomial, or zero-inflated negative ...
3
votes
0answers
382 views

How to handle Zeros in dependent variable in Multiple Linear regression

I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ...
3
votes
0answers
601 views

Overall p-value for zero-inflated beta regression mixed model

I am analysing vegetation percentage cover data from grazed and ungrazed plots in R using a zero-inflated beta regression in package gamlss. Here are some example ...
3
votes
0answers
255 views

Network architecture to predict zero-inflated output

I have a dataset of debt collections and I am trying to predict how much each person paid. The paid amounts are greatly zero-inflated. In the past I have built a two-stage model predicting the ...
3
votes
0answers
160 views

Weird beta inflated distribution qq plot (easy reproducible code included)

First, I do this library(gamlss) simnorm <- rnorm(1000, 0.5,0.1) simzero <- rep(0,1000) x <- sample(c(simnorm,simzero),1000, replace = TRUE) which ...
3
votes
0answers
299 views

Negative binomial regression - only for count data?

I've been pouring over statistics blogs over the last couple days to find a satisfactory answer to this, but I can't seem to find one, so I'll pose it to you in hope of a quick fix. Basically, I'm ...
3
votes
0answers
281 views

Traceplots and gelman.rubin statistics for analysing convergence of mixtures of discrete and continuous distributions

I have a Bayesian hierarchical model which contains a number of distributions which are mixtures of a point-mass at zero and a continuous random variable. The model is fitted using a gibbs sampler. ...
3
votes
0answers
196 views

How to interpret conflicting results between the binomial and count part of a Zero-Inflated Poisson model?

In my model one of my variables (X2 below) is significant and has a negative sign in both the binomial and count parts of a ZIP model. I was wondering how to ...
3
votes
0answers
652 views

glmmADMB- Pseudo R^2 and residual deviance criteria

I’ve been using glmmADMB to fit zero-inflated negative binomial models with a random effect (as far as I can tell, this is the only package that will allow me to do this). I have been trying to do ...
3
votes
0answers
322 views

What do the outputs of my zero inflated poisson model mean?

I am looking a counts of fish within different size classes (0-10, 10-20 etc) between 3 different reef sites, 3 different depths and 2 different survey methods. However, naturally on all observations ...
3
votes
0answers
514 views

Zero-inflated GLMM: correct use of AIC and comparing levels of fixed factor

I am struggling with a ZIGLMM in R. I have a data set on freshwater plant propagules (response variable) and the relation with the ecological state of ponds. Pond state is a categorical factor with ...
3
votes
0answers
507 views

How to Test for Some Basic Assumptions of (Zero Inflated) Negative Binomial?

I am currently working with a T dominant panel --time-series cross-section dataset-- that has N = 8 (eight European countries) and T = 28 (28 quarters for each country). The dependent variable --...
3
votes
0answers
28 views

Estimating proportion of region cropped by proportions in random samples

I have a series of quadrats placed randomly across aerial photography of a region. In each quadrat I have estimated the proportion of the quadrat under cropping and my goal is to estimate the ...
3
votes
0answers
178 views

Appropriate test (in R) for proportion data that aren't normally distributed, aren't based on counts, and include 0's and 1's?

I'm studying differences in tree health among 5 species of trees across 3 different green infrastructure types. Here are the first few lines of data: ...
3
votes
0answers
134 views

Count data with one factor level containing only zeroes

I have a simple poisson glm with one predictor that has three levels. Unfortunately, for one level my response, the variable has only counts of zero. I expected very low counts (perhaps a one or a two ...
3
votes
0answers
797 views

Suitable method for modelling (underdispersed?) count data with lots of zeros and long tail

I have a small data set of counts of bees. I tried a simple Poisson model without random effects but it was very overdispersed (3.95). When I fit a GLMM with random effects (using glmer in lme4) it ...
3
votes
0answers
60 views

How to model continuous predictor variable, $x\ge0$, showing continuity “jump” at $x=0$?

I'm modeling hospital inpatient stays with a logistic regression model, and have encountered a number of predictor variables measuring # of outpatient care episodes which have an inflated number of ...
3
votes
0answers
509 views

Mixed models and normally distributed random effects

I am attempting to analyze some data that includes 10 repeated measurements on 10 different samples. This would normally require using mixed models, and I have attempted to model these using ...
3
votes
0answers
770 views

Is splitting one hurdle model in two GLM/GAM models a valid approach?

I came across several publications dealing with overdispersed zero-inflated count data that "simply" modelled presence absence in one model and then postive counts in a second model. This led to two ...
2
votes
0answers
37 views

How to deal with zero-inflated proportional data in GLMM?

I have proportional data, i.e. number of individuals out of 6 that choose a certain option in a multiple choice experiment, so there are just 7 possible outcomes for each option: 0/6; 1/6; 2/6; 3/6; 4/...
2
votes
0answers
38 views

Choosing error function for regression

I have a dataset with ~100K samples and non-negative continuous target variable. 99% of target values are zeros and the remaining 1% are right-skewed. Here are the deciles (0 and 1 correspond to min ...
2
votes
0answers
61 views

How to conduct a principal component analysis on data set with large number of zeros

I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
2
votes
1answer
106 views

My data has overdispersion but the Hurdle model estimated theta is 0. What am I doing wrong?

I am confused by the dispersion parameter from my model. My data fails the overdispersion test. It's mean is 28.7, the variance is 18655.27. N=2916 of which 32% are zeros. How can theta equal 0 in ...
2
votes
0answers
46 views

Model for semicontinuous data with structural and sampling (true and false) zeros

I am dealing with a very hard-to-work data set: fish larval density. It is a semicontinuous data, with 90% of zeros and a right-skewed distribution, with few very large values. One problem is that ...
2
votes
0answers
103 views

Zero-inflated highly skewed predictor variables

I've thoroughly searched this website and multiple others and can't seem to find an answer to my question. This is also my first post so I hope I've followed all the rules. I apologise for the length, ...
2
votes
0answers
133 views

How to do simple slope analysis for interactions in zero-inflated negative binomial regression in r?

I used the function 'zeroinfl' from the 'pscl' package to fit a zero-inflated model with negative binomial distribution (zinb). Results show significant interaction effects and I should do simple ...
2
votes
0answers
89 views

Zero-inflated Poisson with clustered data in R

I am working with ecology count data with a significant amount of zeroes, and I used a multivariate zero-inflated Poisson regression to evaluate the impact of two independent variables on my dependent ...
2
votes
0answers
65 views

Two sample test clustered data, continuous variable with zeros

I have a situation where an experiment is being run in the following manner: A one stage cluster sampling (I think this is accurate description) is conducted whereby there are multiple organizations ...
2
votes
0answers
41 views

Model and predictor selection in generalized linear models

I’m analyzing count data in R and I want to make two decisions: 1) what type of regression to use (Poisson, negative binomial, zero-inflated, etc), and 2) what predictors to include in the model. I’m ...
2
votes
0answers
1k views

Distribution with zeros that is lognormal without zeros

I have 79 observations with 30 as first quartile, 50 as median, 50.5 as mean, 68 as third quartile. Max value at 169, min value at zero. In particular there are only 4 observations at zero. The ...
2
votes
0answers
193 views

Measure of relationship between two variables that are percentages containing many zeros

I am working with various different data sets (in the context of forest reclamation on industrial disturbed landscapes) that contain percent cover values of desired (planted) and undesired plant ...
2
votes
0answers
126 views

Should I use post-hoc tukey HSD or some other test for pairwise comparisons of a factor on a zero-inflated negative binomial mixed model (ZINB)?

I ran a zero-inflated negative binomial mixed model (ZINB) and now have a statistically significant factor (recording type, either NOCA, BCCH, or Human). To assess the differences between these three ...
2
votes
1answer
135 views

Hierachical cluster analysis of ordinal variables?

I have a dataset containing 400 variables (chemical compounds, amount classes) with values 0,1,2, or 3 and 50 entries (species). Can I use hierarchical cluster analysis to get a dendrogram that ...
2
votes
0answers
96 views

Zero inflated data. ordinal variable prediction. Insurance data

I have an insurance claims data with claim amount as a target variable. The claim amount has a lot of 0s . The output is to predict the high-risk customers. i.e., to preserve the order of claim ...
2
votes
0answers
75 views

Appropriate Model for inflated variable at non zero value

I have a question regarding the appropriate regression model for a continuous variable that ranges from zero to 10. This variable has a large concentration of observations in the "10" value (almost 50 ...
2
votes
0answers
77 views

Which test can I use to compare two count data means with a lot of zeros?

Suppose I have two count datasets, lets say the count of tigers across different years at two places. Now for most of the years we have 0 tiger sightings. For eg. ...
2
votes
0answers
184 views

Revenue based AB test

I have preformed an AB test on revenues (as my target is to increase revenues). Two groups of users were offered with a different marketing offer which in group A (Blue) led to a higher conversion ...
2
votes
0answers
144 views

Sensitivity analysis on constants: logit transformation for ANOVA

I have categorical looking-time data (looks to visually presented items A, B, C and D over a number of seconds), containing 0s and 1s. I want to compare groups (adults, children; n=~30 for each group) ...
2
votes
0answers
174 views

Comparing count data models?

I am trying to fit Negative binomial, Zero Inflated Negative Binomial, Negative Binomial Hurdle, and Random effects negative binomial. Here is the value of AIC for different model: Negative Binomial: ...