# What to do with heterogeneity of variance when spread decreases with larger fitted values

I am trying to produce a linear mixed model the R code is as follows.

lme(Average.payoff~Game+Type+Others.Type+Game:Type+Game:Others.Type+Type:Others.Type,random=~1|Subjects,method="REML", data=Subjectsm1)->lme1

The response term Average.payoff is continuous whilst all explanatory variables are all binary.

When I come to validation I can clearly see that the spread of the residuals decreases with larger fitted values. Although there seems to be alot of information on heterogeneity in the form where the residuals increase with larger fitted values I have read nothing about cases similar to my own.

I have plotted the residuals against each explanatory effect and can see that the spread decreases with larger fitted values for the variables Game and Type but increases for the variable Others.Type.

What is the cause of this and what should I do about it?

Should I look at adding quadratic terms or using additive modelling? Is there a transformation that should be applied?

Thanks,

Jonathan

• @Macro I am think of the models that provide jointly equations for the conditional expectation and the conditional variance of the response variable Y given the covariates. These models are discussed in Wayne Fuller's "Measurement Error Models" and in Carroll et al "Measurement Error in Nonlinear Models: A Modern Perspective." The idea is to express the error component of the model as having a variance that is σ$^2$ multiplied by a nonnegative function g of the covariates in the model. Aug 21 '12 at 14:36