The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

376 questions
Filter by
Sorted by
Tagged with
198 views

Compare two samples with many zeros

We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ...
647 views

What statistics can I use?

I have done a research looking at different frequencies of abrasions (ablation, etc.) over time (in hrs) and my data mainly consists of zeros. As I am weak in statistics, I am unsure which statistics, ...
149 views

Mixed effect zero inflated negative binomial model: “the leading minor of order 1 is not positive definite”

I am having trouble fitting a mixed effect zero inflated negative binomial model to my data using the GLMMadaptive package: ...
940 views

Modelling and interpreting brms output

I do apologize in advance for this might be very basic questions. I am not really familiar with Bayesian statistics and too, unfortunately, this is the very first time I am analysing data in general. ...
41 views

Do zero-inflated models induce selection bias?

Zero-inflated models (e.g., ZI poisson, ZI negative binomial, hurdle) assume two processes for the generation of the observed outcome variable: a process for deciding whether the outcome is zero or ...
309 views

R: GLMM for unbalanced zero-inflated data (glmmTMB)

Study design: I have count data of snails per date, counted over many dates at sites, nested in localities. So, in each locality the snail counts come from several different sites, repeatedly ...
11 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
24 views

Distribution/analysis method for small dataset with many small/zero values

I have a relatively small dataset (160 observations), of which a very large number of values for response variables are zero or very small (e.g., 114/160 values are 0; range 0-4250, with only 11 ...
46 views

Model for semicontinuous data with structural and sampling (true and false) zeros

I am dealing with a very hard-to-work data set: fish larval density. It is a semicontinuous data, with 90% of zeros and a right-skewed distribution, with few very large values. One problem is that ...
84 views

Residuals still zero inflated after running zero-inflated poisson mixed effect model with glmmTMB

I am working with observational data which has a right skew in the dependent variable. This is a mixed effect model with a poisson distribution as based on discrete data. After finding the residuals ...
58 views

Contradiction between zero-inflated poisson model coefficients and graph of the model?

EDIT: Added an reproducible example For one of my models, it seems the coefficients and the graphed out model do not agree. I'm working with adverse effects data, in which intense reactions are rare ...
249 views

Calculate interaction effect confidence intervals in zero-inflated poisson regression

I'm conducting a zero-inflated Poisson regression using the pscl package in R. I've included interaction terms but am having an issue with interpretation. I am assuming an additive effect and summing ...
36 views

Distribution to use in a regression with a positive skewed variable with many zeros (homicide rates)

I want to study the determinants of homicide rates. However, I see when exploring the data that my dependent variable (homicide rates) has many zeros and is positive skewed. Which distribution family /...
36 views

Hurdle Model: Identify Distribution for Second Stage

I have two kinds of dependent variables, both count data. The first one is binary, and the second one an ordinal index based on multiple binary variables (sum). As they are based on coded occurrences ...
62 views

Dependent variable with many zeros in a difference-in-differences model

There is a question with a similar title: How do I estimate a differences in differences model when the dependent variable has many zeros? However, mine is a little different. Let's assume I have a ...
129 views

Regression predictions show far less variance than expected

New to R and fairly new to statistics - appreciate any input. In short, I'm trying to develop a predictive regression model but after fitting the model on training data, the output for my testing ...
57 views

Update a zero-inflated Poisson model to adjust model predictions

I am trying to model out how a clinical metric declines over time with various therapies. I'm a bit new to R and statistics, so appreciate the patience and help. I have two data sets - the first a ...
109 views

Zero-inflation GLMMs: On the use of different sets of explanatory variables in main and ZI formulas

my questions are general in nature so I won't provide any data. For reference: I am using the package glmmTMB in R so if my terminology is weird it is because it is a mix of this and other sources I'...
149 views

Why doesn't the zero model in a hurdle model exactly match logit result?

I am relatively new to R and I suspect that there is user error here, but I cannot figure out why the output from the logit in the hurdle model does not match the prediction of the "zero" function in ...
64 views

using normal Generalized additive model rather than zero inflated regression

I am doing regression analysis for my data , nearly half of my data is Zero . I have conducted Generalized additive model for my data ; but I was wondering if it is enough to do only generalized ...
339 views

Very skewed and zero-inflated continuous outcome variable

I am trying to predict a positive and continuous outcome by using the generalized linear model in R (glm function) and I am wondering what family could I use for the training data. Some of the ...
89 views

Zero-inflated Poisson distribution parameter estimates

Let's say we have a population distributed by Zero-inflated Poisson distribution:  f(x | \psi, \lambda) = \left\{ \begin{array}{ll} 1-\psi + \psi e^{-\lambda} & \mbox{if } x = 0 \\ \psi \...
78 views

Testing for zero inflation and overdispersion in count time series data?

How can I test whether my data is really zero inflated? Can I use the same methods as for count data? Thank you.
43 views

Pooling estimates and variances with zero counts

I have a dataset with 10 different sampling groups. The sampling is done in order to maximize the ability to find events. The sample looks a little like: ...
42 views

Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
86 views

Zero inflated binomial for excessive zeros

"What are some tricks for dealing with a zero inflated response variable when tackling a machine learning regression problem?" Answer : "One of the easiest and most intuitive methods is to run a ...
12 views

Duration Analysis with Clumping at Infinity?

I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...
68 views

Inverting data for Zero-inflated mixed effects models

I am looking for some advice on my analyses, I have been going back and forth between co-authors about the validity of my approach and would appreciate some external input. My data are derived from ...
389 views

Dealing with excessive number of zeros

ipdb> np.count_nonzero(test==0) / len(ytrue) * 100 76.44815766923736 ...
139 views

High proportion of zero values and PCA

My aim is to perform PCA since I have 76 variables in my dataset. Problem is that most of my variables are highly skewed as you can see in the histogram below. These variables are proportions ...
102 views

Zero-inflated highly skewed predictor variables

I've thoroughly searched this website and multiple others and can't seem to find an answer to my question. This is also my first post so I hope I've followed all the rules. I apologise for the length, ...
435 views

GAMM with Zero-Inflated Negative Binomial - Looking for a package on R

I am looking for an R package to fit Generalized Additive Mixed Models with ZINB distribution, as ZINB is not available in the mgcv package nor in the gamm4 package. I read here that it might be ...
129 views

Compute a measure of explained variance for hurdle models in R

I am working with a dataset df which comprises count data count and a number of categorical variables. ...
1k views

Forecasting daily time series sales revenue with many zero entries

I have been trying to forecast the sales revenue of different product groups (the displayed sales revenue is aggregated over all products for each day e.g. smartphones with different prices as one ...
174 views

Zero inflates Negative Binomial difference in difference model

I have data set up in the standard difference in difference format. Each subject has two observations, one for pre and one for post intervention. The outcome variable is total number of ER visits. ...
133 views

How to do simple slope analysis for interactions in zero-inflated negative binomial regression in r?

I used the function 'zeroinfl' from the 'pscl' package to fit a zero-inflated model with negative binomial distribution (zinb). Results show significant interaction effects and I should do simple ...
716 views

Zero-inflated count data simulation in R

I need to simulate data to explore the question: "What happens if my data have zero inflation, but I ignore it and fit a standard count model instead?" More specifically, how can I add zeros in a way ...
38 views

Mean + Variance/Mean ~ 1 for 70 different (Poisson Distributed?) time series variables

I am working with a dataset from my job which is the number of "events" that occur at 70 different locations on a daily basis, over 964 days. So I have univariate panel data. I imagined each location ...
278 views

How to interpret Zero-Inflated Poisson in WINBUGS?

I have Winbugs code for a zero-inflated Poisson (ZIP) model. I obtained this code from my lab at university and the person who wrote it is not accessible for me to ask questions. Here is the code: <...
54 views

How to interpret zero-inflation model for Bayesian regression?

I am trying to understand the zero-inflated poisson (ZIP) model used in Bayesian regression modelling. I came across code here for the ZIP model. My question is related to the 3rd line of code within ...
29 views

What is more useful to find the type of distribution I have, an histogram or a qqplot?

First, I don't know which data to consider when I talk about the distribution of my data. All my data? Or each distribution of the data of all the levels of my conditions? I have tested people three ...
164 views

Deviance residual for zero-inflated Poisson model

I have been trying to explore the residuals associated with zero inflated models for I x J contingency table. I can figure out deviance residuals for ZINB model through zinbwave package. But I don't ...
20 views

Multiple Stage Hurdle Model Analyses

I have a dilemma over the application of double hurdle model. Here is a description of the data that I want to analyze: 1. Binary info on household purchase of different milk products 2. If ...
456 views

Negative binomial model vs zero-inflated negative binomial - theoretical justifications

I have a count variable that I would like to predict using a categorical variable (it has 4 levels). I would like to decide whether I should use Poisson, negative binomial, or zero-inflated negative ...
89 views

Zero-inflated Poisson with clustered data in R

I am working with ecology count data with a significant amount of zeroes, and I used a multivariate zero-inflated Poisson regression to evaluate the impact of two independent variables on my dependent ...
16 views

How to model if a company is mentioned in a review (volume) and how it is mentioned (valence)?

I am working with social media data of Brand A and would like to model what features of posts of Brand A are linked to if a competitor (Brand B) is mentioned in the comments of different posts of ...
104 views

zero inflated poisson iregression AIC values

Im trying to regress count data (DV). There are a few IV's both continuous and categorical. Frequency count of my DV as show below.Two groups , treatment and control. About 75 % are zeros. ...
65 views

Two sample test clustered data, continuous variable with zeros

I have a situation where an experiment is being run in the following manner: A one stage cluster sampling (I think this is accurate description) is conducted whereby there are multiple organizations ...
I have a data set where $100$ people made $500$ trips for $5$ days. I want to build a trip-level regression (zero-inflated Poisson) where the dependent variable will be the count of hard-braking in ...