I have a data set as follows:

Response variable; proportion (i.e., 0.15, 0.94, 0.26) Predictor variables; both fixed, one continuous (400,700,1000,1300), one catagorical (2-levels of species)

I am interested in the interaction between these two predictor variables.Residuals Plot

Code for the model used: glm_necrosis <- glm(necrosis ~ Sp + Treat + Sp*Treat, family = gaussian)

I have run a linear regression in R and plotted my residuals against predicted values and while there is no clear non-linear trend in the data, homogeneity of variance was violated (classic wedge shape). A q-q plot also showed that the data was highly leptokurtic and non-normal. I tried a number of transformations (arcsine, log, squared, etc.), and nothing could improve on the normality or variance assumptions.

Following the arcsine transformation, this is what the residuals looked like.

Arcsine-sqrt Resids Plot

I would like to know what my options might be for continuing. Would a generalized additive model be appropriate, or are there non-parametric linear regression alternatives that might would better?

I appreciate any assistance.

  • $\begingroup$ You can't interpret a QQ plot in the presence of heteroskedasticity, so ignore that as your error is more fundamental. You would expect heteroskedasticity with proportions, from two different causes (only one of which would be improved by transformation). Can you show the residual plot you have this wedge shape in? Are these count-proportions (count on some total count) or continuous proportions (like fraction of land area of a particular type, say)? $\endgroup$
    – Glen_b
    Commented Jun 1, 2017 at 0:30
  • $\begingroup$ Hi Glen_b, the data are continuous proportions (fraction of live tissue remaining in a sample). I will try to add the residuals plot to this post. $\endgroup$
    – JMann
    Commented Jun 1, 2017 at 4:24
  • $\begingroup$ Is the size of each sample you have the fractions for the same? How are these fractions obtained? (some potential methods might effectively work out as count-proportions anyway). Are there exact 0's or 1's? $\endgroup$
    – Glen_b
    Commented Jun 1, 2017 at 4:39
  • $\begingroup$ All samples were approximately equal in size. In order to quantify this variable, I subtracted the area of live tissue from the surface area of the sample and then divided by the total surface area. This gave me the proportion of the tissue that had died during the experimental period. $\endgroup$
    – JMann
    Commented Jun 1, 2017 at 4:45
  • $\begingroup$ 1. how did you calculate area? (counting cells or counting squares for example? via some instrument? some other way) 2. Are there exact 0's or 1's? 3. The residuals look much as you'd expect; (I'm somewhat surprised an arcsin-sqrt didn't show a substantial improvement.). .... I'd be inclined to use beta regression, or 0-1 inflated beta regression if you have exact 1's or 0's, though a quasi-binomial may do for capturing the changing variance $\endgroup$
    – Glen_b
    Commented Jun 1, 2017 at 4:55

1 Answer 1


1) You may consider a logit transform log(p/(1-p)) because you are dealing with proportions (and, by the way, if you know out of how many sample points these proportions are computed, then consider glm with a binomial family for a logistic regression).

2) What are the plotted residuals you used? The variance inhomogeneity may be due to influential points (corresponding to the rightmost points on your residual plots above). You may consider studentized residuals instead. See http://www.statmethods.net/stats/rdiagnostics.html under Non-constant Error Variance.

3) If that does not work and you may also consider the function gamlss from package gamlss. There is a flexible way for fitting inhomogeneous sigma. I don't have enough practical experience to propose a code here though.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.