5
$\begingroup$

I have read one cannot/should not calculate the basic R-Squared used in linear regression for a Poisson generalized linear regression model.

It is logical to me that one cannot determine the basic R-Squared for a logistic regression, since the R-Squared gives the proportion of explained variance of the dependent variable and the dependent variable is not metric in a logistic regression.

However, the dependent variable in a Poisson regression is clearly metric. It might not be fully continuous, but that should not be a requirement to calculate the variance. So I do not understand why one should not calculate the basic R-Squared for a Poisson regression model.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
10
$\begingroup$

A Poisson regression is nonlinear. Yes, it’s called a generalized linear model, but it has a nonlinear link function. Poisson regression is not linear.

When a regression is nonlinear, the residuals and predictions are not orthogonal. This means that the total sum of squares does not decompose into the sum of squares of the regression and the sum of squares of the residuals. There is a third term.

Therefore, $R^2=SSReg/SSTotal$ does not represent the proportion of variance explained.

This has to do with the “other” I mention here: Nonlinear quantile regression SSReg analogue.

EDIT

In the two years since I posted this, I have wrote a question and self-answer on nonlinear regression $R^2$ that might be worth reading. The gist is that $R^2$ is, in some sense, equivalent to mean squared error (MSE). Tying back to Poisson regression, if you’re interested in the MSE, $R^2$ might be reasonable to calculate, though this is not the standard loss function under consideration for Poisson regression (corresponds to maximum likelihood estimation for a Gaussian response, not Poisson).

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Ok, thx a lot for the quick answer! $\endgroup$
    – Benykō-Zamurai
    Commented Jun 28, 2020 at 14:52
  • 1
    $\begingroup$ I think there are some generalizations of the r-squared for logistic regression expressed in terms of ratios of chi-square statistics from the log likelihood. I'm not confident, but my gut says that since the poisson is in the same family, those expressions might also be of some use. $\endgroup$
    – Demetri Pananos
    Commented Jun 28, 2020 at 17:13
  • $\begingroup$ en.wikipedia.org/wiki/Logistic_regression#Pseudo-R2s $\endgroup$
    – Dave
    Commented Jun 28, 2020 at 17:22
3
$\begingroup$

Apparently for a Poisson regression model no R-square but a so-called deviance statistic has been proposed and is computed with many software packages.

Further to quote:

The deviance is a measure of how well the model fits the data - if the model fits well, the observed values $Y_{i}$ will be close to their predicted means $\mu_{i}$, causing both of the terms in D to be small, and so the deviance to be small.

The cited formula is:

For a fitted Poisson regression the deviance is equal to

${D = 2 \sum^{n}_{i=1} \{ Y_{i} \log(Y_{i}/\mu_{i}) - (Y_{i}-\mu_{i}) \}}$

where

${\mu_{i} = \exp(\hat{\beta}_{0} + \hat{\beta}_{1}X_{1} + ... + \hat{\beta}_{p} X_{p})}$

denotes the predicted mean for observation i based on the estimated model parameters.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What about elements where Yi == 0 ? μi is estimated, but Yi is taken from the dataset, right? $\endgroup$
    – Pawel
    Commented Sep 13, 2021 at 13:26
  • $\begingroup$ OK, I found it under the link you have provided: "where if Y_i=0, the Y_{i} \log(Y_{i}/\mu_{i}) term is taken to be zero..." Thanks! $\endgroup$
    – Pawel
    Commented Sep 13, 2021 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.