Skip to main content
51 votes
Accepted

Is a "hurdle model" really one model? Or just two separate, sequential models?

Separating the log-likelihood It is correct that most hurdle models can be estimated separately (I would say, instead of sequentially). The reason is that the log-likelihood can be decomposed into ...
Achim Zeileis's user avatar
31 votes
Accepted

Continuous generalization of the negative binomial distribution

That's an interesting question. My research group has been using the distribution you refer to for some years in our publicly available bioinformatics software. As far as I know, the distribution does ...
Gordon Smyth's user avatar
  • 13.2k
20 votes

Diagnostics for generalized linear (mixed) models (specifically residuals)

This is an old question, but I thought it would be useful to add that option 4 suggested by the OP is now available in the DHARMa R package (available from CRAN, see here). The package makes the ...
Florian Hartig's user avatar
19 votes

Continuous generalization of the negative binomial distribution

Look at this paper: Chandra, Nimai Kumar, and Dilip Roy. A continuous version of the negative binomial distribution. Statistica 72, no. 1 (2012): 81. It's defined in the paper as the survival ...
Aksakal's user avatar
  • 61.9k
19 votes

Quantifying diversity of bird species

Just about any general book on ecological methods has a section on diversity measures and there are indeed several dedicated monographs on diversity in ecology alone, to say nothing about related ...
Nick Cox's user avatar
  • 58.6k
17 votes

How to correctly compute mutual information (Python Example)?

To calculate mutual information, you need to know the distribution of the pair $(X,Y)$ which is counts for each possible value of the pair. This would be described by a 2 dimensional matrix as in ...
Benoit Sanchez's user avatar
15 votes
Accepted

How to formulate the offset of a GLM

I don't know where you heard that a Poisson or negative binomial with an offset is preferable to a binomial model for a number of individuals surviving out of an initial number; I would normally ...
Ben Bolker's user avatar
  • 44.5k
14 votes

Relationship between Poisson, binomial, negative binomial distributions and normal distribution

The binomial distribution is the distribution of the number of successes in a fixed (i.e. not random) number of independent trials with the same probability of success on each trial. It support is ...
Michael Hardy's user avatar
12 votes

Robust estimators for count data

Yes, there are. To name just one, which I've had good experiences with, you can minimize the Cramer-von Mises distance between the empirical distribution and the theoretical distribution with ...
jbowman's user avatar
  • 40.5k
11 votes

Clustering of very skewed, count data: any suggestions to go about (transform etc)?

@ttnphns has provided a good answer. Doing clustering well is often about thinking very hard about your data, so let's do some of that. To my mind, the most fundamental aspect of your data is that ...
gung - Reinstate Monica's user avatar
11 votes

Quantifying diversity of bird species

Firstly, I assume that you are talking about alpha-diversity (although the existence of multiple sites where the data were taken suggests that beta-diversity could be also relevant). The simplest ...
Roger V.'s user avatar
  • 4,439
11 votes

Count predictor and binary outcome

Per your questions... Is a binary logistic regression the best approach when I have a count predictor and a binary outcome? Yes. Logistic regression handles any linear equation which requires the ...
Shawn Hemelstrand's user avatar
10 votes

Robust estimators for count data

If the issue merely boils down to very high or very low observations, one would be tempted to just use a trimmed mean. The problem with that of course, is that your estimate may be biased. You could ...
AdamO's user avatar
  • 63.7k
9 votes

Estimating probability of attack in Ukraine, given count data

This is not an answer, but rather a side comment: Keep in mind that the new attacks are not independent of the previous ones. Historical data is not necessarily relevant for the future. It is probably ...
Tim's user avatar
  • 140k
9 votes

Quantifying diversity of bird species

The other thing that's worth taking into account (although it involves similar deep rabbit holes as the other aspects of diversity metrics mentioned in other answers and comments) is that diversity ...
Ben Bolker's user avatar
  • 44.5k
9 votes
Accepted

Poisson regression with small denominators/counts

First of all, a binomial GLM is more appropriate than a Poisson GLM. (A Poisson GLM is used for unbounded counts; your counts are bounded by the total number of surgeries.) The counts aren't that ...
Rachel Altman's user avatar
9 votes

Standard negative binomial regression when counts are mainly zeros?

I don't quite think that a distinction between "true" and "untrue" ("false"?) zeros is very helpful. Zero inflated distributions arise naturally if your data generating ...
Stephan Kolassa's user avatar
8 votes
Accepted

Small Sample Sizes and Zero Inflated Count Data in R

The problem you are observing with lack of significance of GA has nothing to do with zero inflation or with random effects. It is simply a limitation of Wald tests for count models. If you replace the ...
Gordon Smyth's user avatar
  • 13.2k
8 votes

Count predictor and binary outcome

Is a binary logistic regression the best approach when I have a count predictor and a binary outcome? It is certainly one valid approach, probably the most common one. Is it "best"? That ...
Peter Flom's user avatar
  • 125k
7 votes
Accepted

Count Data - Gaussian, poisson or quasipoisson?

The answer is likely to be quasipoisson. This will depend a bit on how much data you have. Is it only slightly more than the number of parameters (12)? Assuming you have at least, say, 24 counts: ...
JDL's user avatar
  • 1,404
7 votes

Dealing with outliers in dependent variables?

The number of visitors is a counted variable and I would expect it to be highly skewed. A first model to try might be Poisson regression, which is equivalent to working on a log scale (specifically, ...
Nick Cox's user avatar
  • 58.6k
7 votes
Accepted

Why GLM Poisson model predict negative value for count data?

The Poisson GLM fits a model $y_i \sim \text{Pois}(\mu_i)$ with $\log(\mu_i) = x_i^\top \beta$, i.e., a log links the expectation $\mu_i$ to the so-called "linear predictor" $x_i^\top \beta$, often ...
Achim Zeileis's user avatar
7 votes

Overdispersion tests from DHARMa and sjstats: conflicting results?

I'm the developer of DHARMa. First of all: note that results are not actually conflicting - a non-significant test doesn't mean that there is no overdispersion, it just means just that the respective ...
Florian Hartig's user avatar
7 votes
Accepted

Confidence intervals for the mean of a sample of counts

It's a bit nuanced. You could pull out the big guns and use a poisson regression ...
Demetri Pananos's user avatar
7 votes

Modelling count data with extreme underdispersion - what distribution?

The Conway-Maxwell-Poisson model has recently been shown to handle arbitrarily small underdispersion (see Huang 2020). For example, it is possible to have a mean of 15 and a variance of 2, say, by ...
user302932's user avatar
7 votes

Pair-matched count regression in R with offset?

You should be able to do this with a mixed model with a count response (e.g. Poisson or negative binomial): you want the "standard" count-GLM-with-offset model with random variation in the ...
Ben Bolker's user avatar
  • 44.5k
7 votes

Estimating probability of attack in Ukraine, given count data

Does anyone know what kind of model I would use for something like this? ... I was just wondering if anyone know some common approaches. Two approaches you may want to look into: "Self exciting ...
krkeane's user avatar
  • 2,210
7 votes

Theoretical justification for using a zero-inflated count model

The key distinction here between using a zero-inflated model or not is already in this part of your question: It is sometimes useful to conceive of those zeros coming from two different generating ...
Shawn Hemelstrand's user avatar
7 votes

Linear regression when independent variable are count data

OLS regression makes no assumptions about the distribution of the independent variables. It makes assumptions about the errors, which we usually look at through the residuals. The IVs can be ...
Peter Flom's user avatar
  • 125k
7 votes
Accepted

Percentage of females as a regressor in count model with population as offset - interpretation?

The weird magnitudes of the original regression coefficients evidently came from numerical difficulties in maximum-likelihood estimation. It often makes sense to center and scale the values of ...
EdM's user avatar
  • 95.8k

Only top scored, non community-wiki answers of a minimum length are eligible