Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

Filter by
Sorted by
Tagged with
0
votes
2answers
148 views

Regression assumption

I have a data set where $100$ people made $500$ trips for $5$ days. I want to build a trip-level regression (zero-inflated Poisson) where the dependent variable will be the count of hard-braking in ...
4
votes
1answer
789 views

Can you use glmmTMB to simultaneously model offsets and zero-inflation?

I'm currently modelling microbial data, with multiple samples and groups of samples. Two problems arise with my data: 1) The data is zero-inflated and dispersed (large variation); 2) Each sample has a ...
1
vote
0answers
81 views

Dirichlet-Multinomial Distribution with many zero counts

Short version: Is there someway to make the dirchlet-multinomial distribution sensitive to the presence of zero counts? Long version: I am attempting model metric positions in musical data. You can ...
2
votes
0answers
41 views

Model and predictor selection in generalized linear models

I’m analyzing count data in R and I want to make two decisions: 1) what type of regression to use (Poisson, negative binomial, zero-inflated, etc), and 2) what predictors to include in the model. I’m ...
2
votes
2answers
75 views

How do we compare count models for prediction and inference?

I have estimated a number of count models on a data, including Poisson, Zero-Inflated Poisson (ZIP), mixed-effects Poisson, mixed-effects ZIP and, a few different versions of each of these based on ...
1
vote
1answer
164 views

What approach can I use to analyse zero-inflated, overdispersed, count data with very low replicates and a nested random effect?

I am having trouble with analysing some of my data. I'm trying to test the effect of a treatment with two levels (Treatment/Control) on the abundance of ants belonging to different dominance ...
3
votes
1answer
121 views

Classification followed by regression?

I have the following problem: I have a dataset for which my observations have a bunch of features and a continuous response (regression problem). However, some of my observations (about a fourth of ...
2
votes
1answer
43 views

how to model this type of distribution?

I am trying to model this distribution in a generalized mixed model. the variable is a measure of number of number of years, reflecting start to end of reproduction, i.e. reproductive period. This ...
3
votes
0answers
382 views

How to handle Zeros in dependent variable in Multiple Linear regression

I am totally new to machine learning (and to this platform too) and was trying to implement Multiple linear Regression to improve my ranking algorithm. I have a data-set which have the following ...
0
votes
1answer
61 views

Linear Regression with both variables centered on 0

Just a question to be sure, if my dependent variable and independant variable are centered on 0, how do I interpret correctly the linear regression ? The intercept must be significant for the slope ...
2
votes
1answer
242 views

Zero inflation Poisson regression

I am using penalised methods based on glmnet package in R. I used the zero inflated Poisson regression for my sample which contains 2,734 observations and 27 ...
2
votes
0answers
1k views

Distribution with zeros that is lognormal without zeros

I have 79 observations with 30 as first quartile, 50 as median, 50.5 as mean, 68 as third quartile. Max value at 169, min value at zero. In particular there are only 4 observations at zero. The ...
2
votes
0answers
193 views

Measure of relationship between two variables that are percentages containing many zeros

I am working with various different data sets (in the context of forest reclamation on industrial disturbed landscapes) that contain percent cover values of desired (planted) and undesired plant ...
1
vote
1answer
315 views

Generate zero-inflated random vector

I want to randomly generate a vector of size n - that represents hypothetical revenues per users in a webshop. Typically I would imagine this distribution to ...
0
votes
1answer
1k views

hurdle model with non-zero gaussian distribution in R

I have biomass data (continuous response variable). If sufficient data is collected, the log(Biomass) follows a normal distribution. However, I am separating the overall biomass by family (i.e., ...
0
votes
0answers
149 views

How to model wind speed with a weibull distribution when there is excessive zeros in the distribution? [duplicate]

Hey I have a data set that contains hourly wind speed for about seven years. I am trying to model the data using a weibull distribution which according to literature is a good fit for hourly wind ...
1
vote
2answers
58 views

Testing significance between samples with all zeroes in one

I have Count data with zeros, and I'd like to use a poisson GLM (or similar) to compare two groups. One group has all zeros, the other is count data ranging from 0-15. Since one group has all zeros –...
1
vote
0answers
79 views

Random effects in ZIP or ZINB models - count portion, Bernoulli portion, or both?

I'm building some zero-inflated models that need to include random effects, and I can't find a definitive answer for whether I should include the random effects in the part of the model that describes ...
1
vote
0answers
49 views

Fitting zero inflated right truncated negative binomial distribution

I have a dataset that is bounded above by 2000 (physically not possible to have larger than 2000 values). Is there a way, in R, to fit a zero inflated, upper bounded negative binomial distribution to ...
2
votes
1answer
334 views

How to do preprocessing for zero inflated variable in multiple regression?

I am trying to build a multiple regression model, and many of my variables looks like this (histogram for time spent in the system). The reason I had such data is because zero is actually represents ...
0
votes
1answer
31 views

Computing cost-per-use for a zero-use item with sunken cost

The very idea of a cost-per-use (CPU) analysis implies that either all units have use, or that cost is incurred only with use. However, there are situations where neither of these situations hold true....
3
votes
0answers
597 views

Overall p-value for zero-inflated beta regression mixed model

I am analysing vegetation percentage cover data from grazed and ungrazed plots in R using a zero-inflated beta regression in package gamlss. Here are some example ...
0
votes
0answers
93 views

Hurdle model with weighted count variable as dependent variable?

My question concerns the feasibility of applying a hurdle model when my dependent variable is not exactly a count, but a "weighted count". More precisely, my weighted count variable (...
0
votes
1answer
195 views

Posthoc test for zero-altered models (hurdle, ZNAB)

I have created a zero-altered negative binomial model (ZNAB, using the hurdle function in R). This model consists of two parts, one which analysis zero values, and the other analysing positive values ...
0
votes
0answers
89 views

What do two spots mean in the GLM model validation graphs?

I performed a hurdle (delta) model to estimate the relative indices of abundance of a fish species. Thus, the catch estimates involved fitting separately two ‘sub-models’ to the data. The first sub-...
0
votes
0answers
36 views

Comparing several Poisson Models [duplicate]

I have training data that I have fitted with two types of Poisson models (one zero inflated and one standard) and I would like to compare each model's performance on a test set to see which one is ...
25
votes
3answers
4k views

Is a “hurdle model” really one model? Or just two separate, sequential models?

Consider a hurdle model predicting count data y from a normal predictor x: ...
1
vote
1answer
354 views

Likelihood ratio test with zero inflated models to check overdispersion

I am trying to compare ZIP and ZINB models and see if there is an overdispersion. They have the same parameters except alpha. I am wondering if it is correct to use Likelihood ratio test, the same way ...
1
vote
1answer
67 views

Count regression with censored counts that aren't necessarily zero.

Say you have count outcomes $Y_1, ..., Y_n$ out of different population sizes (e.g. offsets in a usual count model) $p_1, ..., p_n$. I want to relate this to a covariate $X_1, ..., X_n$ using a usual ...
2
votes
0answers
126 views

Should I use post-hoc tukey HSD or some other test for pairwise comparisons of a factor on a zero-inflated negative binomial mixed model (ZINB)?

I ran a zero-inflated negative binomial mixed model (ZINB) and now have a statistically significant factor (recording type, either NOCA, BCCH, or Human). To assess the differences between these three ...
2
votes
2answers
4k views

Forecasting daily time series with many zeros

I need to forecast a univariate time-series of sales data with the following characterica. It is a daily time-series Around 70-80 % of the date nothing is sold ($x_t = 0$) At the 20-30 % remaining ...
2
votes
1answer
220 views

Steps of a clustering problems composed by right-skewed data and large number of zeros

I'm trying to cluster a dataset based on 190 diabetic patients and 20 columns (features of patients) and many of these features have most zeros (to understand better, the median of 8 of 20 features is ...
6
votes
1answer
5k views

Interpret zero-inflated negative binomial regression

I am trying to estimate a zero-inflated negative binomial model with 11 predictor variables and the number of reported crimes as a response variable. The model seems to work OK, but I'm uncertain on ...
0
votes
2answers
86 views

sjstats- Model is overfitting zero-counts

I have a glmmTMB model and I used the overdisp and the zero_count functions with these results: ...
0
votes
0answers
668 views

deviance for zero-inflated compound poisson model, continuous data (R)

I am looking at homicide rates per US county in a given 3-yr period as a DV, against several demographic and political measures as IV. I have A LOT of zeros for the DV, and these are presumably real ...
1
vote
0answers
268 views

zero inflated poisson model, how to choose “inflate” variables

I am trying to run a zero-inflated Poisson regression. The data I have are number of West Nile Virus cases NoCases (dependent var), and my independent variables are AvgTemp, AvgPrecipitaion, Region (*...
2
votes
3answers
2k views

How to forecast demand with time series and/or other models?

I need to forecast data which has many periods of zero demand, also there is no seasonality or trend in the data. I tried ARIMA, but it converges to the mean. I also applied some predictors, but ...
5
votes
1answer
3k views

Zero inflated beta regression using gamlss for vegetation cover data

My goal is to analyse vegetation cover data. The way the data collection works is that you throw a quadrat (0.5m x 0.5m) in a sample plot and estimate the percent cover of the target species. Here is ...
2
votes
1answer
135 views

Hierachical cluster analysis of ordinal variables?

I have a dataset containing 400 variables (chemical compounds, amount classes) with values 0,1,2, or 3 and 50 entries (species). Can I use hierarchical cluster analysis to get a dendrogram that ...
3
votes
0answers
255 views

Network architecture to predict zero-inflated output

I have a dataset of debt collections and I am trying to predict how much each person paid. The paid amounts are greatly zero-inflated. In the past I have built a two-stage model predicting the ...
1
vote
1answer
231 views

Transformation of data with 0 values

I've read some of the comments about trying to do log transformations on data that has 0 values. In my data I have two treatments that would be helped with a log transform except each treatment has a ...
1
vote
1answer
757 views

Residuals in Zero-Inflated Negative Binomial Regression

What do residuals mean in the context of zero-inflated negative binomial regression? I'm learning zero-inflated negative binomial regression. The data is from a state education system and includes ...
1
vote
1answer
604 views

Dispersion parameter in mass and PSCL package (zero-inflated negative binomial and negative binomial)

I have run Zero inflated negative binomial and negative binomial model with same data set in R. I get log(theta)= -2.47 for Zero inflated negative binomial and log(theta)= -5.149 for negative binomial ...
0
votes
1answer
404 views

generation of zero-inflated Poisson data in R

I want to generate data from a zero-inflated Poisson distribution in R using the mpath package in the following way: ...
0
votes
1answer
298 views

How to fit Anomaly Detection model to imbalanced time series data(Zero Inflected time series data)?

I'm doing a predictive modeling for predicting anomalies in the sensors data. For this I'm using the twitter AnomalyDetection package in R. We are getting ton of data from sensors for every day. For ...
1
vote
0answers
177 views

Zero in proportion data

I am counting the proportion of seeds that have germinated (RV) in petri dishes. 5 petri dishes each had 25 seeds in them. Treatment (EV) was applied. No seeds germinated. When this is put through the ...
16
votes
0answers
574 views

Fitting custom distributions by MLE

My question relates to fitting custom distributions in R but I feel it has enough of a probability element to remain on CV. I have an interesting set of data which has the following characteristics: ...
1
vote
1answer
65 views

How can I compare two variables when one has lots of zeros?

Specifically, I want to understand the contributions to insurance premiums and claims. In this case, everyone pays a premium, but only some people make claims. I want to understand whether the ...
1
vote
1answer
452 views

Deviance in hurdle model

How can I calculate the deviance for the factors and the null and saturated models with Hurdle models? I used the function hurdle() from the package ...
9
votes
2answers
1k views

GAMM with zero-inflated data

Is it possible to fit a GAMM(Generalized Additive Mixed Model) for zero-inflated data in R? If not, is it possible to fit a GAM(Generalized Additive Model) for zero-inflated data with a negative ...