The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

Filter by
Sorted by
Tagged with
3
votes
3answers
198 views

Compare two samples with many zeros

We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ...
1
vote
0answers
647 views

What statistics can I use?

I have done a research looking at different frequencies of abrasions (ablation, etc.) over time (in hrs) and my data mainly consists of zeros. As I am weak in statistics, I am unsure which statistics, ...
0
votes
1answer
149 views

Mixed effect zero inflated negative binomial model: “the leading minor of order 1 is not positive definite”

I am having trouble fitting a mixed effect zero inflated negative binomial model to my data using the GLMMadaptive package: ...
4
votes
1answer
940 views

Modelling and interpreting brms output

I do apologize in advance for this might be very basic questions. I am not really familiar with Bayesian statistics and too, unfortunately, this is the very first time I am analysing data in general. ...
1
vote
0answers
41 views

Do zero-inflated models induce selection bias?

Zero-inflated models (e.g., ZI poisson, ZI negative binomial, hurdle) assume two processes for the generation of the observed outcome variable: a process for deciding whether the outcome is zero or ...
1
vote
1answer
309 views

R: GLMM for unbalanced zero-inflated data (glmmTMB)

Study design: I have count data of snails per date, counted over many dates at sites, nested in localities. So, in each locality the snail counts come from several different sites, repeatedly ...
0
votes
0answers
11 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
0
votes
0answers
24 views

Distribution/analysis method for small dataset with many small/zero values

I have a relatively small dataset (160 observations), of which a very large number of values for response variables are zero or very small (e.g., 114/160 values are 0; range 0-4250, with only 11 ...
2
votes
0answers
46 views

Model for semicontinuous data with structural and sampling (true and false) zeros

I am dealing with a very hard-to-work data set: fish larval density. It is a semicontinuous data, with 90% of zeros and a right-skewed distribution, with few very large values. One problem is that ...
1
vote
1answer
84 views

Residuals still zero inflated after running zero-inflated poisson mixed effect model with glmmTMB

I am working with observational data which has a right skew in the dependent variable. This is a mixed effect model with a poisson distribution as based on discrete data. After finding the residuals ...
3
votes
1answer
58 views

Contradiction between zero-inflated poisson model coefficients and graph of the model?

EDIT: Added an reproducible example For one of my models, it seems the coefficients and the graphed out model do not agree. I'm working with adverse effects data, in which intense reactions are rare ...
2
votes
2answers
249 views

Calculate interaction effect confidence intervals in zero-inflated poisson regression

I'm conducting a zero-inflated Poisson regression using the pscl package in R. I've included interaction terms but am having an issue with interpretation. I am assuming an additive effect and summing ...
1
vote
1answer
36 views

Distribution to use in a regression with a positive skewed variable with many zeros (homicide rates)

I want to study the determinants of homicide rates. However, I see when exploring the data that my dependent variable (homicide rates) has many zeros and is positive skewed. Which distribution family /...
0
votes
0answers
36 views

Hurdle Model: Identify Distribution for Second Stage

I have two kinds of dependent variables, both count data. The first one is binary, and the second one an ordinal index based on multiple binary variables (sum). As they are based on coded occurrences ...
0
votes
1answer
62 views

Dependent variable with many zeros in a difference-in-differences model

There is a question with a similar title: How do I estimate a differences in differences model when the dependent variable has many zeros? However, mine is a little different. Let's assume I have a ...
3
votes
1answer
129 views

Regression predictions show far less variance than expected

New to R and fairly new to statistics - appreciate any input. In short, I'm trying to develop a predictive regression model but after fitting the model on training data, the output for my testing ...
0
votes
1answer
57 views

Update a zero-inflated Poisson model to adjust model predictions

I am trying to model out how a clinical metric declines over time with various therapies. I'm a bit new to R and statistics, so appreciate the patience and help. I have two data sets - the first a ...
1
vote
1answer
109 views

Zero-inflation GLMMs: On the use of different sets of explanatory variables in main and ZI formulas

my questions are general in nature so I won't provide any data. For reference: I am using the package glmmTMB in R so if my terminology is weird it is because it is a mix of this and other sources I'...
1
vote
1answer
149 views

Why doesn't the zero model in a hurdle model exactly match logit result?

I am relatively new to R and I suspect that there is user error here, but I cannot figure out why the output from the logit in the hurdle model does not match the prediction of the "zero" function in ...
0
votes
1answer
64 views

using normal Generalized additive model rather than zero inflated regression

I am doing regression analysis for my data , nearly half of my data is Zero . I have conducted Generalized additive model for my data ; but I was wondering if it is enough to do only generalized ...
1
vote
1answer
339 views

Very skewed and zero-inflated continuous outcome variable

I am trying to predict a positive and continuous outcome by using the generalized linear model in R (glm function) and I am wondering what family could I use for the training data. Some of the ...
1
vote
0answers
89 views

Zero-inflated Poisson distribution parameter estimates

Let's say we have a population distributed by Zero-inflated Poisson distribution: $$ f(x | \psi, \lambda) = \left\{ \begin{array}{ll} 1-\psi + \psi e^{-\lambda} & \mbox{if } x = 0 \\ \psi \...
0
votes
0answers
78 views

Testing for zero inflation and overdispersion in count time series data?

How can I test whether my data is really zero inflated? Can I use the same methods as for count data? Thank you.
0
votes
0answers
43 views

Pooling estimates and variances with zero counts

I have a dataset with 10 different sampling groups. The sampling is done in order to maximize the ability to find events. The sample looks a little like: ...
0
votes
0answers
42 views

Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
1
vote
0answers
86 views

Zero inflated binomial for excessive zeros

"What are some tricks for dealing with a zero inflated response variable when tackling a machine learning regression problem?" Answer : "One of the easiest and most intuitive methods is to run a ...
0
votes
0answers
12 views

Duration Analysis with Clumping at Infinity?

I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...
1
vote
1answer
68 views

Inverting data for Zero-inflated mixed effects models

I am looking for some advice on my analyses, I have been going back and forth between co-authors about the validity of my approach and would appreciate some external input. My data are derived from ...
0
votes
3answers
389 views

Dealing with excessive number of zeros

ipdb> np.count_nonzero(test==0) / len(ytrue) * 100 76.44815766923736 ...
1
vote
0answers
139 views

High proportion of zero values and PCA

My aim is to perform PCA since I have 76 variables in my dataset. Problem is that most of my variables are highly skewed as you can see in the histogram below. These variables are proportions ...
2
votes
0answers
102 views

Zero-inflated highly skewed predictor variables

I've thoroughly searched this website and multiple others and can't seem to find an answer to my question. This is also my first post so I hope I've followed all the rules. I apologise for the length, ...
1
vote
1answer
435 views

GAMM with Zero-Inflated Negative Binomial - Looking for a package on R

I am looking for an R package to fit Generalized Additive Mixed Models with ZINB distribution, as ZINB is not available in the mgcv package nor in the gamm4 package. I read here that it might be ...
1
vote
1answer
129 views

Compute a measure of explained variance for hurdle models in R

I am working with a dataset df which comprises count data count and a number of categorical variables. ...
3
votes
1answer
1k views

Forecasting daily time series sales revenue with many zero entries

I have been trying to forecast the sales revenue of different product groups (the displayed sales revenue is aggregated over all products for each day e.g. smartphones with different prices as one ...
0
votes
2answers
174 views

Zero inflates Negative Binomial difference in difference model

I have data set up in the standard difference in difference format. Each subject has two observations, one for pre and one for post intervention. The outcome variable is total number of ER visits. ...
2
votes
0answers
133 views

How to do simple slope analysis for interactions in zero-inflated negative binomial regression in r?

I used the function 'zeroinfl' from the 'pscl' package to fit a zero-inflated model with negative binomial distribution (zinb). Results show significant interaction effects and I should do simple ...
4
votes
1answer
716 views

Zero-inflated count data simulation in R

I need to simulate data to explore the question: "What happens if my data have zero inflation, but I ignore it and fit a standard count model instead?" More specifically, how can I add zeros in a way ...
0
votes
1answer
38 views

Mean + Variance/Mean ~ 1 for 70 different (Poisson Distributed?) time series variables

I am working with a dataset from my job which is the number of "events" that occur at 70 different locations on a daily basis, over 964 days. So I have univariate panel data. I imagined each location ...
0
votes
1answer
278 views

How to interpret Zero-Inflated Poisson in WINBUGS?

I have Winbugs code for a zero-inflated Poisson (ZIP) model. I obtained this code from my lab at university and the person who wrote it is not accessible for me to ask questions. Here is the code: <...
1
vote
1answer
54 views

How to interpret zero-inflation model for Bayesian regression?

I am trying to understand the zero-inflated poisson (ZIP) model used in Bayesian regression modelling. I came across code here for the ZIP model. My question is related to the 3rd line of code within ...
0
votes
0answers
29 views

What is more useful to find the type of distribution I have, an histogram or a qqplot?

First, I don't know which data to consider when I talk about the distribution of my data. All my data? Or each distribution of the data of all the levels of my conditions? I have tested people three ...
0
votes
0answers
164 views

Deviance residual for zero-inflated Poisson model

I have been trying to explore the residuals associated with zero inflated models for I x J contingency table. I can figure out deviance residuals for ZINB model through zinbwave package. But I don't ...
1
vote
0answers
20 views

Multiple Stage Hurdle Model Analyses

I have a dilemma over the application of double hurdle model. Here is a description of the data that I want to analyze: 1. Binary info on household purchase of different milk products 2. If ...
3
votes
0answers
456 views

Negative binomial model vs zero-inflated negative binomial - theoretical justifications

I have a count variable that I would like to predict using a categorical variable (it has 4 levels). I would like to decide whether I should use Poisson, negative binomial, or zero-inflated negative ...
2
votes
0answers
89 views

Zero-inflated Poisson with clustered data in R

I am working with ecology count data with a significant amount of zeroes, and I used a multivariate zero-inflated Poisson regression to evaluate the impact of two independent variables on my dependent ...
1
vote
0answers
16 views

How to model if a company is mentioned in a review (volume) and how it is mentioned (valence)?

I am working with social media data of Brand A and would like to model what features of posts of Brand A are linked to if a competitor (Brand B) is mentioned in the comments of different posts of ...
0
votes
1answer
104 views

zero inflated poisson iregression AIC values

Im trying to regress count data (DV). There are a few IV's both continuous and categorical. Frequency count of my DV as show below.Two groups , treatment and control. About 75 % are zeros. ...
2
votes
0answers
65 views

Two sample test clustered data, continuous variable with zeros

I have a situation where an experiment is being run in the following manner: A one stage cluster sampling (I think this is accurate description) is conducted whereby there are multiple organizations ...
4
votes
2answers
553 views

Is a distribution still considered right-skewed if the majority of responses are zero?

i have a distribution in which the majority of cases take the value of zero and then there are a few (perhaps 10%) with values of 1,2 or 3. would this distribution still count as right skewed even ...
0
votes
2answers
148 views

Regression assumption

I have a data set where $100$ people made $500$ trips for $5$ days. I want to build a trip-level regression (zero-inflated Poisson) where the dependent variable will be the count of hard-braking in ...