4
$\begingroup$

I am searching for a good criterion to measure the "goodness of fit" in generalized linear models. To make clear: I am not searching for a criterion which gives me an answer to the question "does overdispersion occur?". What do you think about Nagelkerke's pseudo R-squared? Any thought would be appreciated!

$\endgroup$
  • 2
  • $\begingroup$ Hey Thanks! But I am not looking for a Goodness of fit measurement for logit models! $\endgroup$ – MarkDollar Jun 5 '11 at 14:16
  • $\begingroup$ I am searching for the same thing like Mark and found that there is an adjusted D² (similar to adjusted R²) which can be used to evaluate the goodness of fit for GLMs. It can be calculated using null and residual deviance (see Guisan & Zimmermann, 2000, Ecol Modelling) but I am not sure if it is really correct to use it (because until now it was used only really rarely in literature). Does anyone of you ever has ever used this adjusted D²? Thanks for your answer & best regars, Inga S. $\endgroup$ – user8370 Jan 6 '12 at 12:47
  • $\begingroup$ Hi Inga: Please submit your question by following the "Ask Question" link at the top right of the page. $\endgroup$ – whuber Jan 6 '12 at 14:08
  • $\begingroup$ @linga - I just posted this related question regarding interpretation of percent of deviance explained to this list a day or two before your question. stats.stackexchange.com/questions/20583/… Your $R^2$ deviance (and the $R^2_{GLM}$ in the post by probabilityislogic below ( which is really just percent of deviance explained) is also known as McFadden's pseudo-$R^2$, one of the usual pseudo $R^2$ measures. $\endgroup$ – Brett Jan 6 '12 at 15:56
2
$\begingroup$

You can make a $R^2$ type quantity, by simply noting what $R^2$ is for normal OLS, but in the framework of an exponential family. An exponential family likelihood (with dispersion) can be written as follows

$$f(y_i|\mu_i,\phi)=\exp\left(\frac{y_ib(\mu_i)-c(\mu_i)}{\phi}+d(y_i,\phi)\right)$$

Where $b(.),c(.),d(.;.)$ are known functions. For normal OLS, we have $b(\mu_i)=\mu_i$, $c(\mu_i)=\frac{1}{2}\mu_i^2$ and $d(y_i,\phi)=-\frac{1}{2}\left(log(2\pi\phi)+\frac{1}{\phi}y_i^2\right)$. A goodness of fit test for each observation, or residual, can be obtained by using the scaled likelihood ratio test

$$d_i^2=2\phi\left(log[f(y_i|\mu_i=y_i,\phi)]-log[f(y_i|\mu_i=\hat{\mu}_i,\phi)]\right)$$

$$=2\left[y_ib(y_i)-y_ib(\hat{\mu}_i)-c(y_i)+c(\hat{\mu}_i)\right]$$

This means that, in the OLS case, the squared deviance residual is given by:

$$d_i^2=2\left[y_i^2-y_i\hat{\mu}_i-\frac{1}{2}y_i^2+\frac{1}{2}\hat{\mu}_i^2\right]=[y_i-\hat{\mu}_i]^2=e_i^2$$

Which is just the ordinary squared residual. For OLS, we have $R^2=1-\frac{SSE}{SST}$ where SSE is the sum of squared residuals from the fitted model, and SST is the sum of squared residuals from the intercept only model. Hence, we can analogously define $R^2$ for GLMs as:

$$R^2_{GLM}=1-\frac{\sum_id_{i,model}^2}{\sum_id_{i,null}^2}$$

$\endgroup$
1
$\begingroup$

It will depend on what kind of GLM you're using and your data. For example, the Wald chi-square and likelihood test are good statistics for categorical data.

$\endgroup$
  • $\begingroup$ Hello! I'm not searching for tests that can compare nested models (you mentioned LR Tests). And I'm not searching for quasi t-tests (like Wald Test) to look for significance of the coefficients! I'm searching for something which is based on the (deviance) residuals, something like an adj. R^2 for linear models. I just want to say something like, "This model describes 70% of the deviation". $\endgroup$ – MarkDollar Jun 6 '11 at 8:27

protected by whuber Jan 6 '12 at 16:30

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.