What is the expected distribution of residuals in a generalized linear model?

Question

I am performing a generalized linear model, where I have to specify a family different from the normal one.

• What is the expected distribution of residuals?
• For example, should the residuals be distributed normally?

Answer 1

What is the expected distribution of residuals?

It varies with the model in ways that make this impossible to answer generally.

For example, should the residuals be distributed normally?

Not generally, no.

Answer 2

There is a whole cottage industry centered around designing residuals for GLMs that are more symmetric or even approximately "normal" (i.e. Gaussian), e.g. Pearson residuals, Anscombe residuals, (adjusted) deviance residuals, etc. See for example Chapter 6 of James W. Hardin and Joseph M. Hilbe (2007) "Generalized Linear Models and Extensions" second edition. College Station, TX: Stata Press. If the dependent variable is discrete (an indicator variable or a count) then it is obviously very hard to make the expected distribution of the residuals exactly Gaussian.

One thing you can do is repeatedly simulate new data under the assumption that your model is true, estimate your model using that simulated data and compute the residuals, and then compare your actual residuals with your simulated residuals. In Stata I would do this like so:

sysuse nlsw88, clear
glm wage i.union grade c.ttl_exp##c.ttl_exp, link(log) family(poisson)

// collect which observations were used in estimation and the predicted mean
gen byte touse = e(sample)
predict double mu if touse

// predict residuals
predict resid if touse, anscombe

// prepare variables for plotting a cumulative distribution function
cumul resid, gen(c)

// collect the graph command in the local macro `graph'
local graph "twoway"

// create 19 simulations:
gen ysim = .
forvalues i = 1/19 {
    replace ysim = rpoisson(mu) if touse
    glm ysim i.union grade c.ttl_exp##c.ttl_exp, link(log) family(poisson)
    predict resid`i' if touse, anscombe
    cumul resid`i', gen(c`i')
    local graph "`graph' line c`i' resid`i', sort lpattern(solid) lcolor(gs8) ||"
}
local graph "`graph' line c resid, sort lpattern(solid) lcolor(black) "

// display the graph
`graph' legend(order(20 "actual residuals" 1 "simulations"))