101

Here is what I usually like doing (for illustration I use the overdispersed and not very easily modelled quine data of pupil's days absent from school from MASS): Test and graph the original count data by plotting observed frequencies and fitted frequencies (see chapter 2 in Friendly) which is supported by the vcd package in R in large parts. For ...


54

Recall that an offset is just a predictor variable whose coefficient is fixed at 1. So, using the standard setup for a Poisson regression with a log link, we have: $$\log \mathrm{E}(Y) = \beta' \mathrm{X} + \log \mathcal{E}$$ where $\mathcal{E}$ is the offset/exposure variable. This can be rewritten as $$\log \mathrm{E}(Y) - \log \mathcal{E} = \beta' \...


30

Most important is the logic behind the model. Your variable "number of patents per year" is a count variable, so Poisson regression is indicated. That is a GLM (generalized linear model) with (usually) log link function, while the usual linear regression is a Gaussian GLM with identity link. Here, it is truly the log link function which is most important, ...


29

This is the appearance you expect of such a plot when the dependent variable is discrete. Each curvilinear trace of points on the plot corresponds to a fixed value $k$ of the dependent variable $y$. Every case where $y=k$ has a prediction $\hat{y}$; its residual--by definition--equals $k-\hat{y}$. The plot of $k-\hat{y}$ versus $\hat{y}$ is obviously a ...


29

You have discovered an intimate, but generic, property of GLMs fit by maximum likelihood. The result drops out once one considers the simplest case of all: Fitting a single parameter to a single observation! One sentence answer: If all we care about is fitting separate means to disjoint subsets of our sample, then GLMs will always yield $\hat\mu_j = \bar ...


26

I don't think the title of your question accurately captures what you're asking for. The question of how to interpret the parameters in a GLM is very broad because the GLM is a very broad class of models. Recall that a GLM models a response variable $y$ that is assumed to follow a known distribution from the exponential family, and that we have chosen an ...


25

This is almost a duplicate; the linked question explains that you shouldn't expect the coefficient estimates, residual deviance, nor degrees of freedom to change. The only thing that changes when moving from Poisson to quasi-Poisson is that a scale parameter that was previously fixed to 1 is computed from some estimate of residual variability/badness-of-fit ...


24

This is a great question. We know models such as logistic, Poisson, etc. fall under the umbrella of generalized linear models. Well, yes and no. Given the context of the question, we must be quite careful to specify what we're talking about -- and "logistic" and "Poisson" alone are insufficient to describe what is intended. (i) "Poisson" is a ...


24

Offsets can be used in any regression model, but they are much more common when working with count data for your response variable. An offset is just a variable that is forced to have a coefficient of $1$ in the model. (See also this excellent CV thread: When to use an offset in a Poisson regression?) When used correctly with count data, this will let ...


23

In order to see what is going on here, it is useful first to do the regression without the intercept, since an intercept in a categorical regression with just one predictor is meaningless: > rs1 = glm(breaks ~ tension-1, data=warpbreaks, family="poisson") > rs2 = glm.nb(breaks ~ tension-1, data=warpbreaks) Since Poisson and negative binomial ...


23

The quasi-Poisson is not a full maximum likelihood (ML) model but a quasi-ML model. You just use the estimating function (or score function) from the Poisson model to estimate the coefficients, and then employ a certain variance function to obtain suitable standard errors (or rather a full covariance matrix) to perform inference. Hence, glm() does not supply ...


22

The distinction is clear as soon as you understand what a Pearson residual is. You are correct that for a Poisson model, variance increases as mean increases. As a result, ordinary raw residuals ($r_i=y_i-\hat\mu_i$) should have a spread that increases with fitted values (though not in proportion). However, Pearson residuals are residuals divided by the ...


22

No, there is no general count data regression model. (Just as there is no general regression model for continuous data. A linear model with normally distributed homoskedastic noise is most commonly assumed, and fitted using Ordinary Least Squares. However, gamma regression or exponential regression is often used to deal with different error distribution ...


21

It is possible indeed. As explained at https://methodology.psu.edu/AIC-vs-BIC, "BIC penalizes model complexity more heavily. The only way they should disagree is when AIC chooses a larger model than BIC." If your goal is to identify a good predictive model, you should use the AIC. If your goal is to identify a good explanatory model, you should use the BIC. ...


19

I think the problem that's confusing you is that you're used to having an additive error. Most models won't. Think of linear regression not as a linear mean with an additive error, but as the response being conditionally normal: $$(Y|X) \sim \operatorname{N}(X\beta,\sigma^2I)$$ Then the similarities to GLMs, in particular, to Poisson regression and ...


17

Of course you are correct that the Poisson distribution technically is defined only for integers. However, statistical modeling is the art of good approximations ("all models are wrong"), and there are times when it makes sense to treat non-integer data as though it were [approximately] Poisson. For example, if you send out two observers to record the ...


17

When trying to determine what sort of glm equation you want to estimate, you should think about plausible relationships between the expected value of your target variable given the right hand side (rhs) variables and the variance of the target variable given the rhs variables. Plots of the residuals vs. the fitted values from from your Normal model can help ...


17

You can get the probability of zero-inflation by p <- predict(object, ..., type = "zero") and the mean of the count distribution by lambda <- predict(object, ..., type = "count") See Appendix C of vignette("countreg", package = "pscl") for a few more details. To simulate the distribution, you can either do it manually with ifelse(rbinom(n, size = ...


16

I suspect that part of the problem may lie in your choice of performance metric. If you measure test performance using RMSE then training the model to minimise the MSE matches the test criterion, giving a hint as to what is considered important. You may find that if you measure the test performance using the negative log-likelihood of the test set using a ...


14

10 is large... 1.01 is not. Since the variance of a $\chi^2_k$ is $2k$ (see Wikipedia), the standard deviation of a $\chi^2_k$ is $\sqrt{2k}$, and that of $\chi^2_k/k$ is $\sqrt{2/k}$. That's your measuring stick: for $\chi^2_{100}$, 1.01 is not large, but 2 is large (7 s.d.s away). For $\chi^2_{10,000}$, 1.01 is OK, but 1.1 is not (7 s.d.s away).


14

For a response $y$, if you assume the logarithm of its expectation is a linear combination of predictors $\renewcommand{\vec}[1]{\boldsymbol{#1}}\vec{x}$ $$\operatorname{E}Y_i=\exp{\vec\beta^{\mathrm{T}}\vec{x}_i}$$ & its variance is equal to its expectation $$\operatorname{Var}Y_i=\operatorname{E}Y_i$$ then consistent estimates for the regression ...


14

For the approach of using standard diagnostic plots but wanting to know what they should look like, I like the paper: Buja, A., Cook, D. Hofmann, H., Lawrence, M. Lee, E.-K., Swayne, D.F and Wickham, H. (2009) Statistical Inference for exploratory data analysis and model diagnostics Phil. Trans. R. Soc. A 2009 367, 4361-4383 doi: 10.1098/rsta.2009.0120 ...


14

In principle, I actually agree that 99% of the time, it's better to just use the more flexible model. With that said, here are two and a half arguments for why you might not. (1) Less flexible means more efficient estimates. Given that variance parameters tend to be less stable than mean parameters, your assumption of fixed mean-variance relation may ...


13

This is an old question, but I thought it would be useful to add that my DHARMa R package (available from CRAN, see here) now provides standardized residuals for GLMs and GLMMs, based on a simulation approach similar to what is suggested by @GregSnow. From the package description: The DHARMa package uses a simulation-based approach to create readily ...


13

Assume that you're fitting a multinomial regression model with J categories where the contrast between j and the last category J is modeled as $$ \log \frac{\mu_{ij}}{\mu_{iJ}} = \alpha_j + X_i \beta_j $$ where $X_i$ is a vector of covariates associated with the $i$th case. This is rather a hard optimisation problem because the parameters are coupled by ...


12

A one-inflated Poisson model for a count $Y_i$ is $$\begin{align}\Pr(Y_i = 1) &= \pi_i +(1-\pi_i)\cdot\mu_i\mathrm{e}^{-\mu_i}\\ \Pr(Y_i = y_i) &= (1-\pi_i)\cdot\frac{\mu_i^{y_i}\mathrm{e}^{-\mu_i}}{y_i!} \qquad \text{when } y_i\neq 1 \end{align}$$ where the Poisson mean $\mu_i$ & Bernoulli probability $\pi_i$ are related to the predictors ...


12

Both the Poisson distribution and the geometric distribution are special cases of the negative binomial (NB) distribution. One common notation is that the variance of the NB is $\mu + 1/\theta \cdot \mu^2$ where $\mu$ is the expectation and $\theta$ is responsible for the amount of (over-)dispersion. Sometimes $\alpha = 1/\theta$ is also used. The Poisson ...


12

In Poisson regression there are two Deviances. The Null Deviance shows how well the response variable is predicted by a model that includes only the intercept (grand mean). And the Residual Deviance is −2 times the difference between the log-likelihood evaluated at the maximum likelihood estimate (MLE) and the log-likelihood for a "saturated model" (a ...


11

As far as I can see there's nothing wrong with your code or calculations. You could skip a few lines of code, though, by getting the incidence rate ratios by ${\tt exp(coef(mod))}$. The two models make different assumptions, and this potentially leads to different results. Poisson regression assumes constant hazards. The Cox model only assumes that the ...


11

Poisson regression is just a GLM: People often speak of the parametric rationale for applying Poisson regression. In fact, Poisson regression is just a GLM. That means Poisson regression is justified for any type of data (counts, ratings, exam scores, binary events, etc.) when two assumptions are met: 1) the log of the mean-outcome is a linear combination ...


Only top voted, non community-wiki answers of a minimum length are eligible