Tag Info

Accepted

Is a "hurdle model" really one model? Or just two separate, sequential models?

Separating the log-likelihood It is correct that most hurdle models can be estimated separately (I would say, instead of sequentially). The reason is that the log-likelihood can be decomposed into ...
• 15.2k
Accepted

Diagnostics for generalized linear (mixed) models (specifically residuals)

This answer is not based on my knowledge but rather quotes what Bolker et al. (2009) wrote in an influential paper in the journal Trends in Ecology and Evolution. Since the article is not open access (...
• 5,556
Accepted

Continuous generalization of the negative binomial distribution

That's an interesting question. My research group has been using the distribution you refer to for some years in our publicly available bioinformatics software. As far as I know, the distribution does ...
• 12.1k
Accepted

Identical coefficients estimated in Poisson vs Quasi-Poisson model

This is almost a duplicate; the linked question explains that you shouldn't expect the coefficient estimates, residual deviance, nor degrees of freedom to change. The only thing that changes when ...
• 41.4k

What regression model is the most appropriate to use with count data?

No, there is no general count data regression model. (Just as there is no general regression model for continuous data. A linear model with normally distributed homoskedastic noise is most commonly ...
• 117k

Continuous generalization of the negative binomial distribution

Look at this paper: Chandra, Nimai Kumar, and Dilip Roy. A continuous version of the negative binomial distribution. Statistica 72, no. 1 (2012): 81. It's defined in the paper as the survival ...
• 60k

Quantifying diversity of bird species

Just about any general book on ecological methods has a section on diversity measures and there are indeed several dedicated monographs on diversity in ecology alone, to say nothing about related ...
• 53.9k

Diagnostics for generalized linear (mixed) models (specifically residuals)

This is an old question, but I thought it would be useful to add that option 4 suggested by the OP is now available in the DHARMa R package (available from CRAN, see here). The package makes the ...
• 7,729

How to correctly compute mutual information (Python Example)?

To calculate mutual information, you need to know the distribution of the pair $(X,Y)$ which is counts for each possible value of the pair. This would be described by a 2 dimensional matrix as in ...
• 7,967

Clustering of very skewed, count data: any suggestions to go about (transform etc)?

It is not wise to transform the variables individually because they belong together (as you noticed) and to do k-means because the data are counts (you might, but k-means is better to do on continuous ...
• 56.5k
Accepted

How to formulate the offset of a GLM

I don't know where you heard that a Poisson or negative binomial with an offset is preferable to a binomial model for a number of individuals surviving out of an initial number; I would normally ...
• 41.4k
Accepted

Correlation between continuous data and count data

I'd say there are at least 3 decent options that would make sense for you: Polyserial Correlation - This would be the most exotic of the 3 options and involves an approximation of a latent, ...
• 810

Relationship between Poisson, binomial, negative binomial distributions and normal distribution

The binomial distribution is the distribution of the number of successes in a fixed (i.e. not random) number of independent trials with the same probability of success on each trial. It support is ...
• 8,796

Robust estimators for count data

Yes, there are. To name just one, which I've had good experiences with, you can minimize the Cramer-von Mises distance between the empirical distribution and the theoretical distribution with ...
• 37.6k

Clustering of very skewed, count data: any suggestions to go about (transform etc)?

@ttnphns has provided a good answer. Doing clustering well is often about thinking very hard about your data, so let's do some of that. To my mind, the most fundamental aspect of your data is that ...
Accepted

Forecasting Poisson, accuracy and prediction intervals

You have what is called intermittent demand, that is, a demand time series characterized by "many" zeros. (If your time series is not demand per se, most of what follows will still apply.) So a web ...
• 117k

What regression model is the most appropriate to use with count data?

The "default", the most commonly used and described, distribution of choice for count data is the Poisson distribution. Most often it is illustrated using example of its first practical usage: A ...
• 135k

Robust estimators for count data

If the issue merely boils down to very high or very low observations, one would be tempted to just use a trimmed mean. The problem with that of course, is that your estimate may be biased. You could ...
• 60.4k

Estimating probability of attack in Ukraine, given count data

This is not an answer, but rather a side comment: Keep in mind that the new attacks are not independent of the previous ones. Historical data is not necessarily relevant for the future. It is probably ...
• 135k

Quantifying diversity of bird species

Firstly, I assume that you are talking about alpha-diversity (although the existence of multiple sites where the data were taken suggests that beta-diversity could be also relevant). The simplest ...
• 3,590

Quantifying diversity of bird species

The other thing that's worth taking into account (although it involves similar deep rabbit holes as the other aspects of diversity metrics mentioned in other answers and comments) is that diversity ...
• 41.4k
Accepted

Poisson regression with small denominators/counts

First of all, a binomial GLM is more appropriate than a Poisson GLM. (A Poisson GLM is used for unbounded counts; your counts are bounded by the total number of surgeries.) The counts aren't that ...
Accepted

Appropriate application of Poisson regression?

A couple of thoughts that may or may not help below. It's a bit hard for us to be helpful without seeing your actual data... First of all, your data rather obviously does not follow a standard ...
• 117k

How to do an ANOVA when your data are non-normal with possibly differing variances?

I notice that your response is called "Number of Organisms", and that all the values are non-negative integers. I suspect these are count data. They should not be treated as normally ...

How to do an ANOVA when your data are non-normal with possibly differing variances?

The data, as indicated by the variable names in the linked-to spreadsheet, pertains to number of organisms in 13 groups, so are some kind of count data. We could do better here if we knew some more ...
• 74.6k

Dealing with outliers in dependent variables?

The number of visitors is a counted variable and I would expect it to be highly skewed. A first model to try might be Poisson regression, which is equivalent to working on a log scale (specifically, ...
• 53.9k
Accepted

Why GLM Poisson model predict negative value for count data?

The Poisson GLM fits a model $y_i \sim \text{Pois}(\mu_i)$ with $\log(\mu_i) = x_i^\top \beta$, i.e., a log links the expectation $\mu_i$ to the so-called "linear predictor" $x_i^\top \beta$, often ...
• 15.2k
Accepted

Confidence intervals for the mean of a sample of counts

It's a bit nuanced. You could pull out the big guns and use a poisson regression ...
• 33.4k