Should I give more weight to goodness of fit or to conceptual approach?. Example

Question

I am running mixed effects models with percentage data.

I run my model using a gaussian distribution approach. AIC=-258, my conditional and marginal pseudo-R squares were 0.33 and 0.11 respectively (very good!). I realized that I should model it using a binomial distribution because I have percentages. Now, the results are pretty similar, but my AIC=2386 is worse and pseudo-R squares diminished a lot (0.07 conditional and 0.02 marginal).

Is this saying that the gaussian approach fits the data better and therefore I should use it preferentially? How could I justify it?

Answer 1

There are a bunch of issues here

• You can't compare likelihoods / deviance / AIC between models with continuous vs. count data, see, e.g. Can WAIC be used to compare Bayesian linear regression models with different likelihoods?.

• Moreover, do you have discrete k/n or continuous proportions? In either case, applying an lm on the raw data is usually not a good idea (see What are the issues with using percentage outcome in linear regression?), at least use a transformation (e.g. logit is sometimes used for continuous proportions, also arcsine, but see https://www.ncbi.nlm.nih.gov/pubmed/21560670). Better use a glm, for k/n binomial, for continuous proportions it is common to use beta regression or a pseudo-binomial.

About the decision what to do - in doubt, I would simply go for the data-generating model, i.e. in case of k/n for a binomial. Check the model fit, e.g. with DHARMa, and with k/n binomial, you also have to check for overdispersion.