# Generalized Additive Model or non-parametric linear regression?

I have a data set as follows:

Response variable; proportion (i.e., 0.15, 0.94, 0.26) Predictor variables; both fixed, one continuous (400,700,1000,1300), one catagorical (2-levels of species)

I am interested in the interaction between these two predictor variables. Code for the model used: glm_necrosis <- glm(necrosis ~ Sp + Treat + Sp*Treat, family = gaussian)

I have run a linear regression in R and plotted my residuals against predicted values and while there is no clear non-linear trend in the data, homogeneity of variance was violated (classic wedge shape). A q-q plot also showed that the data was highly leptokurtic and non-normal. I tried a number of transformations (arcsine, log, squared, etc.), and nothing could improve on the normality or variance assumptions.

Following the arcsine transformation, this is what the residuals looked like. I would like to know what my options might be for continuing. Would a generalized additive model be appropriate, or are there non-parametric linear regression alternatives that might would better?

I appreciate any assistance.

• You can't interpret a QQ plot in the presence of heteroskedasticity, so ignore that as your error is more fundamental. You would expect heteroskedasticity with proportions, from two different causes (only one of which would be improved by transformation). Can you show the residual plot you have this wedge shape in? Are these count-proportions (count on some total count) or continuous proportions (like fraction of land area of a particular type, say)? Jun 1, 2017 at 0:30
• Hi Glen_b, the data are continuous proportions (fraction of live tissue remaining in a sample). I will try to add the residuals plot to this post. Jun 1, 2017 at 4:24
• Is the size of each sample you have the fractions for the same? How are these fractions obtained? (some potential methods might effectively work out as count-proportions anyway). Are there exact 0's or 1's? Jun 1, 2017 at 4:39
• All samples were approximately equal in size. In order to quantify this variable, I subtracted the area of live tissue from the surface area of the sample and then divided by the total surface area. This gave me the proportion of the tissue that had died during the experimental period. Jun 1, 2017 at 4:45
• 1. how did you calculate area? (counting cells or counting squares for example? via some instrument? some other way) 2. Are there exact 0's or 1's? 3. The residuals look much as you'd expect; (I'm somewhat surprised an arcsin-sqrt didn't show a substantial improvement.). .... I'd be inclined to use beta regression, or 0-1 inflated beta regression if you have exact 1's or 0's, though a quasi-binomial may do for capturing the changing variance Jun 1, 2017 at 4:55