The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

157 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
574 views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
2k views

Gamma hurdle model for continuous response

I am modelling invertebrate.biomass ~ habitat.type * calendar.day + habitat.type * calendar.day ^ 2, with a random intercept of transect.id (50 transects were repeated 5 times) My response is zero-...
5k views

“Zero-inflated” predictors in regression?

I know that zero-inflated models (e.g. zero-inflated Poisson or negative binomial models) can be used for dependent variables. I also know that in general there are no assumptions for the independent ...
618 views

Does there exist zero-inflated linear regression?

I have a non-count data with huge number of zeros in the target variable. I need to fit a model being a mixture of Dirac delta function and normal distribution parametrized by mean $X\beta$ and ...
626 views

Gibbs sampling deriving complete conditionals with mixture priors

My question is about the derivation of the complete conditionals for Gibbs sampling in a hierarchical model where some of the parameters are mixtures of point-masses and Normal distributions. The ...
3k views

Zero-inflated model: non-finite value supplied by optim

So I have the following model predicting the presence of an animal on a certain spot. As a time unit quarter is initially used, but for one of the species of animals there is some (little) interesting ...
458 views

Suppose I want to do bayesian inference on the regression problem $\beta$ for Y = X$\beta$ + $\epsilon$ for $\epsilon_i \sim N(0,\sigma^2)$. The complication is that the prior for each component $\... 0answers 461 views Basic idea of zero inflated two part models(hurdel) and application to big data (machine learning) I'm currently working on the data which has 90% 0s in response variable. Based on my research, it seems zero inflated models could be a solution to this. However, while I was reading related documents,... 0answers 9k views Fitting a zero-inflated negative binomial regression with R In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ... 3answers 198 views Compare two samples with many zeros We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ... 1answer 58 views Contradiction between zero-inflated poisson model coefficients and graph of the model? EDIT: Added an reproducible example For one of my models, it seems the coefficients and the graphed out model do not agree. I'm working with adverse effects data, in which intense reactions are rare ... 0answers 456 views Negative binomial model vs zero-inflated negative binomial - theoretical justifications I have a count variable that I would like to predict using a categorical variable (it has 4 levels). I would like to decide whether I should use Poisson, negative binomial, or zero-inflated negative ... 0answers 382 views How to handle Zeros in dependent variable in Multiple Linear regression I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ... 0answers 601 views Overall p-value for zero-inflated beta regression mixed model I am analysing vegetation percentage cover data from grazed and ungrazed plots in R using a zero-inflated beta regression in package gamlss. Here are some example ... 0answers 255 views Network architecture to predict zero-inflated output I have a dataset of debt collections and I am trying to predict how much each person paid. The paid amounts are greatly zero-inflated. In the past I have built a two-stage model predicting the ... 0answers 160 views Weird beta inflated distribution qq plot (easy reproducible code included) First, I do this library(gamlss) simnorm <- rnorm(1000, 0.5,0.1) simzero <- rep(0,1000) x <- sample(c(simnorm,simzero),1000, replace = TRUE) which ... 0answers 299 views Negative binomial regression - only for count data? I've been pouring over statistics blogs over the last couple days to find a satisfactory answer to this, but I can't seem to find one, so I'll pose it to you in hope of a quick fix. Basically, I'm ... 0answers 281 views Traceplots and gelman.rubin statistics for analysing convergence of mixtures of discrete and continuous distributions I have a Bayesian hierarchical model which contains a number of distributions which are mixtures of a point-mass at zero and a continuous random variable. The model is fitted using a gibbs sampler. ... 0answers 196 views How to interpret conflicting results between the binomial and count part of a Zero-Inflated Poisson model? In my model one of my variables (X2 below) is significant and has a negative sign in both the binomial and count parts of a ZIP model. I was wondering how to ... 0answers 652 views glmmADMB- Pseudo R^2 and residual deviance criteria I’ve been using glmmADMB to fit zero-inflated negative binomial models with a random effect (as far as I can tell, this is the only package that will allow me to do this). I have been trying to do ... 0answers 322 views What do the outputs of my zero inflated poisson model mean? I am looking a counts of fish within different size classes (0-10, 10-20 etc) between 3 different reef sites, 3 different depths and 2 different survey methods. However, naturally on all observations ... 0answers 514 views Zero-inflated GLMM: correct use of AIC and comparing levels of fixed factor I am struggling with a ZIGLMM in R. I have a data set on freshwater plant propagules (response variable) and the relation with the ecological state of ponds. Pond state is a categorical factor with ... 0answers 507 views How to Test for Some Basic Assumptions of (Zero Inflated) Negative Binomial? I am currently working with a T dominant panel --time-series cross-section dataset-- that has N = 8 (eight European countries) and T = 28 (28 quarters for each country). The dependent variable --... 0answers 28 views Estimating proportion of region cropped by proportions in random samples I have a series of quadrats placed randomly across aerial photography of a region. In each quadrat I have estimated the proportion of the quadrat under cropping and my goal is to estimate the ... 0answers 178 views Appropriate test (in R) for proportion data that aren't normally distributed, aren't based on counts, and include 0's and 1's? I'm studying differences in tree health among 5 species of trees across 3 different green infrastructure types. Here are the first few lines of data: ... 0answers 134 views Count data with one factor level containing only zeroes I have a simple poisson glm with one predictor that has three levels. Unfortunately, for one level my response, the variable has only counts of zero. I expected very low counts (perhaps a one or a two ... 0answers 797 views Suitable method for modelling (underdispersed?) count data with lots of zeros and long tail I have a small data set of counts of bees. I tried a simple Poisson model without random effects but it was very overdispersed (3.95). When I fit a GLMM with random effects (using glmer in lme4) it ... 0answers 60 views How to model continuous predictor variable,$x\ge0$, showing continuity “jump” at$x=0\$?

I'm modeling hospital inpatient stays with a logistic regression model, and have encountered a number of predictor variables measuring # of outpatient care episodes which have an inflated number of ...
509 views

Mixed models and normally distributed random effects

I am attempting to analyze some data that includes 10 repeated measurements on 10 different samples. This would normally require using mixed models, and I have attempted to model these using ...
770 views

Is splitting one hurdle model in two GLM/GAM models a valid approach?

I came across several publications dealing with overdispersed zero-inflated count data that "simply" modelled presence absence in one model and then postive counts in a second model. This led to two ...
37 views

How to deal with zero-inflated proportional data in GLMM?

I have proportional data, i.e. number of individuals out of 6 that choose a certain option in a multiple choice experiment, so there are just 7 possible outcomes for each option: 0/6; 1/6; 2/6; 3/6; 4/...
38 views

Choosing error function for regression

I have a dataset with ~100K samples and non-negative continuous target variable. 99% of target values are zeros and the remaining 1% are right-skewed. Here are the deciles (0 and 1 correspond to min ...
61 views

How to conduct a principal component analysis on data set with large number of zeros

I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
106 views

My data has overdispersion but the Hurdle model estimated theta is 0. What am I doing wrong?

I am confused by the dispersion parameter from my model. My data fails the overdispersion test. It's mean is 28.7, the variance is 18655.27. N=2916 of which 32% are zeros. How can theta equal 0 in ...
46 views

Model for semicontinuous data with structural and sampling (true and false) zeros

I am dealing with a very hard-to-work data set: fish larval density. It is a semicontinuous data, with 90% of zeros and a right-skewed distribution, with few very large values. One problem is that ...
103 views

Zero-inflated highly skewed predictor variables

I've thoroughly searched this website and multiple others and can't seem to find an answer to my question. This is also my first post so I hope I've followed all the rules. I apologise for the length, ...
133 views

How to do simple slope analysis for interactions in zero-inflated negative binomial regression in r?

I used the function 'zeroinfl' from the 'pscl' package to fit a zero-inflated model with negative binomial distribution (zinb). Results show significant interaction effects and I should do simple ...
89 views

Zero-inflated Poisson with clustered data in R

I am working with ecology count data with a significant amount of zeroes, and I used a multivariate zero-inflated Poisson regression to evaluate the impact of two independent variables on my dependent ...
65 views

Two sample test clustered data, continuous variable with zeros

I have a situation where an experiment is being run in the following manner: A one stage cluster sampling (I think this is accurate description) is conducted whereby there are multiple organizations ...
41 views

Model and predictor selection in generalized linear models

I’m analyzing count data in R and I want to make two decisions: 1) what type of regression to use (Poisson, negative binomial, zero-inflated, etc), and 2) what predictors to include in the model. I’m ...
1k views

Distribution with zeros that is lognormal without zeros

I have 79 observations with 30 as first quartile, 50 as median, 50.5 as mean, 68 as third quartile. Max value at 169, min value at zero. In particular there are only 4 observations at zero. The ...
193 views

Measure of relationship between two variables that are percentages containing many zeros

I am working with various different data sets (in the context of forest reclamation on industrial disturbed landscapes) that contain percent cover values of desired (planted) and undesired plant ...
126 views

Should I use post-hoc tukey HSD or some other test for pairwise comparisons of a factor on a zero-inflated negative binomial mixed model (ZINB)?

I ran a zero-inflated negative binomial mixed model (ZINB) and now have a statistically significant factor (recording type, either NOCA, BCCH, or Human). To assess the differences between these three ...
135 views

Hierachical cluster analysis of ordinal variables?

I have a dataset containing 400 variables (chemical compounds, amount classes) with values 0,1,2, or 3 and 50 entries (species). Can I use hierarchical cluster analysis to get a dendrogram that ...
96 views

Zero inflated data. ordinal variable prediction. Insurance data

I have an insurance claims data with claim amount as a target variable. The claim amount has a lot of 0s . The output is to predict the high-risk customers. i.e., to preserve the order of claim ...
75 views

Appropriate Model for inflated variable at non zero value

I have a question regarding the appropriate regression model for a continuous variable that ranges from zero to 10. This variable has a large concentration of observations in the "10" value (almost 50 ...
77 views

Which test can I use to compare two count data means with a lot of zeros?

Suppose I have two count datasets, lets say the count of tigers across different years at two places. Now for most of the years we have 0 tiger sightings. For eg. ...
184 views

Revenue based AB test

I have preformed an AB test on revenues (as my target is to increase revenues). Two groups of users were offered with a different marketing offer which in group A (Blue) led to a higher conversion ...