28
$\begingroup$

I am trying to fit data with a GLM (poisson regression) in R. When I plotted the residuals vs the fitted values, the plot created multiple (almost linear with a slight concave curve) "lines". What does this mean?

library(faraway)
modl <- glm(doctorco ~ sex + age + agesq + income + levyplus + freepoor + 
                freerepa + illness + actdays + hscore + chcond1 + chcond2,
                family=poisson, data=dvisits)
plot(modl)

enter image description here

$\endgroup$
7
  • $\begingroup$ I don't know if you can upload the plot (sometimes newcomers can't), but if not, could you at least add some data & R code to your question so people can evaluate it? $\endgroup$ Commented Mar 22, 2012 at 15:46
  • $\begingroup$ Jocelyn, I've updated your post with information you put in a comment. I also tagged this as homework since you talked about an assignment. $\endgroup$
    – chl
    Commented Mar 22, 2012 at 16:20
  • $\begingroup$ try plot(jitter(mod1)) to see if the graph is a bit more readable. Why don't you define residuals for us and give us your best guess as interpreting the graph yourself. $\endgroup$ Commented Mar 22, 2012 at 16:45
  • 1
    $\begingroup$ From the question, I'm going to assume that you understand the Poisson distribution & Pois reg, and what a plot of residuals vs fitted values tells you (update if that's wrong), thus you are just wondering about the odd appearance of the points in the plot. B/c this is homework, we don't quite answer as our general policy, but provide hints. I notice that you have a lot of covariates, I wonder if you have 1 continuous & many binary covariates. $\endgroup$ Commented Mar 22, 2012 at 17:45
  • 1
    $\begingroup$ Two followups from gung's comment. First, try table(dvisits$doctorco). What do the 10 curved lines on your plot correspond to, in this table? Also, with in excess of 5000 observations, don't worry too much about fitting 13 regression coefficients. $\endgroup$
    – guest
    Commented Mar 22, 2012 at 22:07

2 Answers 2

35
$\begingroup$

This is the appearance you expect of such a plot when the dependent variable is discrete.

Each curvilinear trace of points on the plot corresponds to a fixed value $k$ of the dependent variable $y$. Every case where $y=k$ has a prediction $\hat{y}$; its residual--by definition--equals $k-\hat{y}$. The plot of $k-\hat{y}$ versus $\hat{y}$ is obviously a line with slope $-1$. In Poisson regression, the x-axis is shown on a log scale: it is $\log(\hat{y})$. The curves now bend down exponentially. As $k$ varies, these curves rise by integral amounts. Exponentiating them gives a set of quasi-parallel curves. (To prove this, the plot will be explicitly constructed below, separately coloring the points by the values of $y$.)

We can reproduce the plot in question quite closely by means of a similar but arbitrary model (using small random coefficients):

# Create random data for a random model.
set.seed(17)
n <- 2^12                       # Number of cases
k <- 12                         # Number of variables
beta = rnorm(k, sd=0.2)         # Model coefficients
x <- matrix(rnorm(n*k), ncol=k) # Independent values
y <- rpois(n, lambda=exp(-0.5 + x %*% beta + 0.1*rnorm(n)))

# Wrap the data into a data frame, create a formula, and run the model.
df <- data.frame(cbind(y,x))    
s.formula <- apply(matrix(1:k, nrow=1), 1, function(i) paste("V", i+1, sep=""))
s.formula <- paste("y ~", paste(s.formula, collapse="+"))
modl <- glm(as.formula(s.formula), family=poisson, data=df)

# Construct a residual vs. prediction plot.
b <- coefficients(modl)
y.hat <- x %*% b[-1] + b[1]     # *Logs* of the predicted values
y.res <- y - exp(y.hat)         # Residuals
colors <- 1:(max(y)+1)          # One color for each possible value of y
plot(y.hat, y.res, col=colors[y+1], main="Residuals v. Fitted")

Residuals vs. fitted

$\endgroup$
7
  • 8
    $\begingroup$ (+1) The color goes a long way in showing what is happening. $\endgroup$
    – cardinal
    Commented Jun 11, 2012 at 23:11
  • 1
    $\begingroup$ So is the above plot concerning? Texts (Statistical Modeling for Biomedical Researchers: A Simple Introduction to the Analysis of Complex Data, Dupont, 2002, p. 316, e.g.) indicate the fitted vs. residual plot should be centered about the zero residual line, and either fan (if raw residuals) or not (if deviance, e.g.). With a limited range of counts in the outcome variable, you get these bands, and, as in the above plot, they're not centered about the line at y = 0. How do we know the OP's residual plot (or the example plot made in this answer) indicates the model is fitting the data well? $\endgroup$
    – Meg
    Commented Aug 10, 2016 at 20:37
  • 1
    $\begingroup$ @Meg That advice doesn't directly apply to residuals of a GLM. Note that the model used to illustrate this answer is known to be correct because it's the one used to generate the data. $\endgroup$
    – whuber
    Commented Aug 10, 2016 at 22:12
  • $\begingroup$ 1/2: Thanks @whuber. I understand for this answer the model is known to be correct since the data were simulated from a given distribution, but in practice it's unknown (as in the OP's post). Also, what I wrote about residuals does apply to POI regression (not all GLMs, no, but this one) - the reference I gave was discussing POI regression specifically. I've only seen texts show standardized POI residuals (Pearson or deviance, e.g.) centered about y = 0, so I'm not sure what I should be looking for, because for this model (which is obviously correct), the plot looks nothing like that. $\endgroup$
    – Meg
    Commented Aug 11, 2016 at 0:15
  • $\begingroup$ 2/2: Do you have a go-to reference that discusses POI residuals more thoroughly by any chance? $\endgroup$
    – Meg
    Commented Aug 11, 2016 at 0:16
10
$\begingroup$

Sometimes stripes like these in residual plots represent points with (almost) identical observed values that get different predictions. Look at your target values: how many unique values are they? If my suggestion is correct there should be 9 unique values in your training data set.

$\endgroup$
5
  • 2
    $\begingroup$ +1. (There's actually a tenth one represented as a single point near the upper right corner.) The values, of course, are $0, 1, \ldots, 9$. $\endgroup$
    – whuber
    Commented Jun 11, 2012 at 20:54
  • $\begingroup$ does it mean that there is no real relationship between target and predictor(s) ? Or some factors in this multiple linear regression can input great noise ? Perhaps, some kind of [Exploratory factor analysis (EFA) ](en.wikipedia.org/wiki/Factor_analysis) and Confirmatory Factor Analysis should be used before regressing ? or just correct grouping (by clusters) will help ? $\endgroup$
    – JeeyCi
    Commented Oct 2 at 13:47
  • $\begingroup$ ... I don't like PCA (with its factor loadings ) for its unconveniency of interpretation of PCA's results $\endgroup$
    – JeeyCi
    Commented Oct 2 at 13:49
  • $\begingroup$ Anyway, I would better rearrange somehow input data (if target is really iid ?), just cannot imagine how. Perhaps some knowledge from partial dependencies analysis can help somehow ? $\endgroup$
    – JeeyCi
    Commented Oct 2 at 14:04
  • $\begingroup$ !! really, the knowledge was here: ". This pattern can occur if you fail to account for fixed/random effects in your model and the fixed effects are correlated with explanatory variables." $\endgroup$
    – JeeyCi
    Commented Oct 2 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.