Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [zero-inflation]

Excessive 0's in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle (2-part) models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB).

0
votes
0answers
8 views
0
votes
0answers
24 views

Is my data inappropriate for a zero-inflated regression model?

I am working with count data where I have an abundance of zeros for one of my categorical factors (Day). I have generated two models, p1 and m1, with zeroinfl() and ...
3
votes
1answer
32 views

Zero-inflated model predicting only a small range of values. I need help

I built a ZI model and it is producing predicted values that are from a very small range when compared to the observed values. Plus it does not produce any zeros. See the fitted vs. observed graph ...
0
votes
0answers
12 views

Variable importance when performing zero-inflated Poisson regression in R?

In short, I need to get the importance of the variables after a zero-inflated regression, with all my predictors being dichotomous factors. I tried something like this: ...
0
votes
0answers
8 views

Yes/no GZLMM zero inflated

I have got a variable which I want to analyse using a GZLMM with binomial distribution as the variable is coded as yes/no(1/0). However, there are a lot of zeros and not many 1's.I was hoping to ...
0
votes
0answers
11 views

What coefficients to include in logit component of zero-inflated and hurdle models?

I'm new to statistics so hoping for a ELI5 explanation! I need to use a hurdle (or zero-inflated) model to try and replicate someone elses methodology on a newer dataset for my undergraduate ...
2
votes
1answer
73 views

My data has overdispersion but the Hurdle model estimated theta is 0. What am I doing wrong?

I am confused by the dispersion parameter from my model. My data fails the overdispersion test. It's mean is 28.7, the variance is 18655.27. N=2916 of which 32% are zeros. How can theta equal 0 in ...
0
votes
0answers
25 views

Plotting a single chart with zero inflated model in R [closed]

I'm new to statistics and R. Currently going through UCLA's zero-inflated model example. The plot from the example outputs to this: I'm wondering how to adapt the code so there is no need for a facet ...
0
votes
0answers
7 views

Different significance found in zero-inflated negative binomial vs. binomial logistic regression

I'm having a hard time reconciling two seemingly contradictory findings: In a binomial logistic regression (where 0 is abstinent and 1 is relapsed), the 7 category nominal predictor showed ...
1
vote
0answers
23 views

GLM Frequency and Severity Models. How Do I improve from here? (R code) [closed]

Background: I've been tasked with creating a rating model by Peril using GLMs. It's commercial lines property, so the data is pretty sparse. The carriers have been asking for Premiums by peril, so we'...
0
votes
1answer
46 views

Zero-inflated vs not Zero-inflated models for count data

I am analyzing a small dataset (d) of urinary track infections in a group of residents of a long-term care institution over a period of 6 months. The total number of patients was 29. I had used ...
0
votes
0answers
14 views

Handling quasi-perfect separation in a zero-inflated negative binomial regression in R

I want to run a zero-inflated negative binomial regression in R, but one of my variables exhibits quasi-complete separation and throws errors for both the negative binomial and logistic pieces. I've ...
0
votes
0answers
17 views

Using gamm4 on zero-inflated count data with Tweedie or zero-inflated Poisson distributions

I'm working with a dataset with a large number of zero-counts on the response variable. This dataset consists of qualitatively coded interviews in a number of important categories, but many of the ...
3
votes
2answers
50 views

GLMMs for count data with glmmTMB: random slopes specification, cross-level-interaction and strange results

folks, I recently found the great glmmTMB package which I hoped would help me with my models. My data are 60,000 facebook posts that are nested in 51 companies (i.e., the posts by these companies). ...
1
vote
1answer
47 views

General linear mixed model in R which will fit quasi family [closed]

I am trying to run a GLMM with a quasibinomial family (my data is 0 inflated and I have a negative min x value), but am receiving this error message as quasi families cannot be used in glmer: ...
1
vote
1answer
91 views

Trouble modeling zero-inflated data. Estimates and standard errors are off with GLM, GLMM, and ZI models

I conducted a study looking at the attraction of different species of insects to 5 different chemical treatments (I have had other issues with this dataset explored here and here). This experiment ...
0
votes
0answers
13 views

Zero-Inflated Rare Keyword Impact Prediction

Suppose we have a dataset $X$ of features and a target binary prediction $y\in\{0,1\}$ for each datapoint. Each row of $X$ consists of counts (a bag of features). We can normalize each row of $X$ to ...
3
votes
3answers
54 views

Compare two samples with many zeros

We carried out a number of some experiments and got 10 independent 2-samples datasets. Is it possible to show a significant difference between the two samples, if each of them contains more than 75% ...
1
vote
0answers
106 views

What statistics can I use?

I have done a research looking at different frequencies of abrasions (ablation, etc.) over time (in hrs) and my data mainly consists of zeros. As I am weak in statistics, I am unsure which statistics, ...
0
votes
1answer
39 views

Mixed effect zero inflated negative binomial model: “the leading minor of order 1 is not positive definite”

I am having trouble fitting a mixed effect zero inflated negative binomial model to my data using the GLMMadaptive package: ...
3
votes
1answer
165 views

Modelling and interpreting brms output

I do apologize in advance for this might be very basic questions. I am not really familiar with Bayesian statistics and too, unfortunately, this is the very first time I am analysing data in general. ...
0
votes
0answers
23 views

Do zero-inflated models induce selection bias?

Zero-inflated models (e.g., ZI poisson, ZI negative binomial, hurdle) assume two processes for the generation of the observed outcome variable: a process for deciding whether the outcome is zero or ...
1
vote
1answer
77 views

R: GLMM for unbalanced zero-inflated data (glmmTMB)

Study design: I have count data of snails per date, counted over many dates at sites, nested in localities. So, in each locality the snail counts come from several different sites, repeatedly ...
0
votes
0answers
10 views

Independent variables with an important share of zeros

In a linear panel data model, is it an issue to have explanatory variables with an important share of zeros (e.g. 40% of observations are zeros)? Can the coefficients of an OLS regression be biased?
0
votes
0answers
20 views

Distribution/analysis method for small dataset with many small/zero values

I have a relatively small dataset (160 observations), of which a very large number of values for response variables are zero or very small (e.g., 114/160 values are 0; range 0-4250, with only 11 ...
2
votes
0answers
35 views

Model for semicontinuous data with structural and sampling (true and false) zeros

I am dealing with a very hard-to-work data set: fish larval density. It is a semicontinuous data, with 90% of zeros and a right-skewed distribution, with few very large values. One problem is that ...
1
vote
1answer
40 views

Residuals still zero inflated after running zero-inflated poisson mixed effect model with glmmTMB

I am working with observational data which has a right skew in the dependent variable. This is a mixed effect model with a poisson distribution as based on discrete data. After finding the residuals ...
2
votes
0answers
37 views

Contradiction between zero-inflated poisson model coefficients and graph of the model?

EDIT: Added an reproducible example For one of my models, it seems the coefficients and the graphed out model do not agree. I'm working with adverse effects data, in which intense reactions are rare ...
2
votes
2answers
88 views

Calculate interaction effect confidence intervals in zero-inflated poisson regression

I'm conducting a zero-inflated Poisson regression using the pscl package in R. I've included interaction terms but am having an issue with interpretation. I am assuming an additive effect and summing ...
1
vote
1answer
27 views

Distribution to use in a regression with a positive skewed variable with many zeros (homicide rates)

I want to study the determinants of homicide rates. However, I see when exploring the data that my dependent variable (homicide rates) has many zeros and is positive skewed. Which distribution family /...
0
votes
0answers
23 views

Hurdle Model: Identify Distribution for Second Stage

I have two kinds of dependent variables, both count data. The first one is binary, and the second one an ordinal index based on multiple binary variables (sum). As they are based on coded occurrences ...
0
votes
1answer
31 views

Dependent variable with many zeros in a difference-in-differences model

There is a question with a similar title: How do I estimate a differences in differences model when the dependent variable has many zeros? However, mine is a little different. Let's assume I have a ...
3
votes
1answer
100 views

Regression predictions show far less variance than expected

New to R and fairly new to statistics - appreciate any input. In short, I'm trying to develop a predictive regression model but after fitting the model on training data, the output for my testing ...
0
votes
1answer
32 views

Update a zero-inflated Poisson model to adjust model predictions

I am trying to model out how a clinical metric declines over time with various therapies. I'm a bit new to R and statistics, so appreciate the patience and help. I have two data sets - the first a ...
1
vote
1answer
54 views

Zero-inflation GLMMs: On the use of different sets of explanatory variables in main and ZI formulas

my questions are general in nature so I won't provide any data. For reference: I am using the package glmmTMB in R so if my terminology is weird it is because it is a mix of this and other sources I'...
1
vote
1answer
65 views

Why doesn't the zero model in a hurdle model exactly match logit result?

I am relatively new to R and I suspect that there is user error here, but I cannot figure out why the output from the logit in the hurdle model does not match the prediction of the "zero" function in ...
0
votes
1answer
42 views

using normal Generalized additive model rather than zero inflated regression

I am doing regression analysis for my data , nearly half of my data is Zero . I have conducted Generalized additive model for my data ; but I was wondering if it is enough to do only generalized ...
0
votes
1answer
124 views

Very skewed and zero-inflated continuous outcome variable

I am trying to predict a positive and continuous outcome by using the generalized linear model in R (glm function) and I am wondering what family could I use for the training data. Some of the ...
1
vote
0answers
57 views

Zero-inflated Poisson distribution parameter estimates

Let's say we have a population distributed by Zero-inflated Poisson distribution: $$ f(x | \psi, \lambda) = \left\{ \begin{array}{ll} 1-\psi + \psi e^{-\lambda} & \mbox{if } x = 0 \\ \psi \...
0
votes
0answers
52 views

Testing for zero inflation and overdispersion in count time series data?

How can I test whether my data is really zero inflated? Can I use the same methods as for count data? Thank you.
0
votes
0answers
37 views

Pooling estimates and variances with zero counts

I have a dataset with 10 different sampling groups. The sampling is done in order to maximize the ability to find events. The sample looks a little like: ...
0
votes
0answers
32 views

Reduce the effect of excessive zeros

I am working on an autoregression problem where I use sequential LSTM. My target is well defined, but I think I am facing a problem with the features. As the features were non-stationary, then I ...
1
vote
0answers
50 views

Zero inflated binomial for excessive zeros

"What are some tricks for dealing with a zero inflated response variable when tackling a machine learning regression problem?" Answer : "One of the easiest and most intuitive methods is to run a ...
0
votes
0answers
6 views

Duration Analysis with Clumping at Infinity?

I am currently trying to build a model to analyze how price setting today affects how long it takes for a customer to return. My first cut was to fit a Weibull regression where the log of the scale ...
1
vote
1answer
63 views

Inverting data for Zero-inflated mixed effects models

I am looking for some advice on my analyses, I have been going back and forth between co-authors about the validity of my approach and would appreciate some external input. My data are derived from ...
0
votes
3answers
215 views

Dealing with excessive number of zeros

ipdb> np.count_nonzero(test==0) / len(ytrue) * 100 76.44815766923736 ...
1
vote
0answers
73 views

High proportion of zero values and PCA

My aim is to perform PCA since I have 76 variables in my dataset. Problem is that most of my variables are highly skewed as you can see in the histogram below. These variables are proportions ...
2
votes
0answers
67 views

Zero-inflated highly skewed predictor variables

I've thoroughly searched this website and multiple others and can't seem to find an answer to my question. This is also my first post so I hope I've followed all the rules. I apologise for the length, ...
1
vote
1answer
228 views

GAMM with Zero-Inflated Negative Binomial - Looking for a package on R

I am looking for an R package to fit Generalized Additive Mixed Models with ZINB distribution, as ZINB is not available in the mgcv package nor in the gamm4 package. I read here that it might be ...
1
vote
1answer
63 views

Compute a measure of explained variance for hurdle models in R

I am working with a dataset df which comprises count data count and a number of categorical variables. ...