I am running a linear mixed model and want to check its assumptions. The model I run is comparing if males and females behave differently over time (timeclass=1,2,3,4):

 x <- lme(response ~ timeclass*sex, random = ~ 1|subject,  method="ML", data=dat)

The code to create the plots:

plot(x, response ~ fitted(.) | sex, abline = c(0,1))
hist((resid(x) - mean(resid(x), na.rm=T)) / sd(resid(x), na.rm=T), freq=F); curve(dnorm, add = TRUE)

This is the output I get:

residual plot linear relationship males and females qqplot for sex qqplot for timeclass histogram

The plots except the residual plot seem to show that there seems to be a linear relationship between the fitted values per sex and the response and that the errors are relatively normally distributed in both sex and across the four time classes. However, the first plot has a very clear diamond shape, almost so clear that it seems values are cut off in the four triangular angles.

Can someone help me understand what is going on and what this means? Is the model I run valid based on the plots for assumptions?


2 Answers 2


The diamond pattern in the residuals is due to a combination of the "ceiling effect" you have in the observed female data (i.e., a lot of female data points clustered at the maximum value of 18 or so) and the "floor effect" you have in the observed male data (a lot of male data points clustered at the minimum value of 0).

To see this, imagine taking your second figure (the one with the female and male data plotted side by side in different panels), overlaying the two panels into a single combined plot, and then rotating this plot 45 degrees clockwise so that the identity line is now horizontal. You would get something pretty close to the top graph.

  • $\begingroup$ Thanks for your comment Jake, that makes sense. If I remember correctly, one of the assumptions of a linear model is that the response variable should be continuos, which can be relaxed if the response is on a reasonably fine scale and if we can assume that the scale is homogeneous. Can I assume my model has passed the assumptions or is the lower variance at the lower and upper limits a problem? If so, what solution would I have? Transforming the data would not solve that problem. $\endgroup$
    – crazjo
    Commented Dec 2, 2013 at 9:22
  • $\begingroup$ The ceiling and floor effects (also called 'censoring') could be a problem, yes, but unfortunately not a straightforward one to fix. There are methods of dealing with censored data such as Tobit regression, but I'm not sure what their extensions to mixed models look like. Sorry I can't be of more help. $\endgroup$ Commented Dec 10, 2013 at 10:25

Diamond (double outward box) distributed residuals are another type of non–monotonic heteroscedasticity that severely violates the homoscedasticity assumption. For the correction of diamond type of heterocedasticity, the study " Correcting Double Outward Box Distributed Residuals by WCEV" (http://www.tandfonline.com/doi/full/10.1080/03610926.2016.1213289) can be helpful.

(Update: 2018-01-02)
The test is so simple and powerful. The test based on auxiliary simple regression of absolute or squared residuals on centered external variable. Test independent from numbers of explanatory variables. Usual tests are monotonic. This test also can detect non monotonic patterns such as diamond, butterfly types of heteroskedasticity. The article also contains SAS codes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.