GEE vs LME, non-normal distribution

Question

I have a question about statistics theory.

I have longitudinal, repeated measures data where the response variable is skewed right. Using R, I ran a linear mixed-effects model (good for longitudinal, repeated measures data that is normally distributed). I obtained an MSE of 0.034. I then ran the same data using generalized estimating equations (good for longitudinal, repeated measures data and does not assume normal distribution). I obtained an MSE of 0.094. The GEE model seems more appropriate for my data because it does not assume normal distribution but results in a higher MSE than LME which does assume normal distribution.

Can we say that LME is overfitting the data? Is there another possible explanation?

This is how the 2 models visualize:

Answer 1

A couple of points:

• The estimating equation for the fixed-effects coefficients $\beta$ is the same under a GEE and under a linear mixed model. In a sense, the idea behind the GEEs is to adapt the score equation from a mixed model to work for categorical responses borrowing ideas from GLMs. The difference between the two approaches is on how the covariance matrix of the multivariate outcome is estimated. In the linear mixed model you use (restricted) maximum likelihood, whereas in the GEE you use a method of moments estimator based on the residuals, coupled with a working assumption for the correlations that is corrected for misspecification using the sandwich estimator.
• Hence, even though strictly speaking you don't assume any distribution in GEEs, you see that in your specific case the difference lies in the assumption that you make for the variance-covariance matrix of your outcome. That is, in general, the GEE approach aims to protect you against misspecification of the covariance structure. However, this is not "free-of-charge," you pay for it in terms of power. Namely, if your assumed covariance structure under the mixed model is correct, then the mixed model is more efficient than the GEE.
• Another relevant and important point is about missing data. The GEE will give you valid inferences if the missing data mechanism is missing completely at random, whereas the linear mixed model under the missing at random mechanism. The latter is more reasonable to hold in practice than the former one.

Answer 2

Expanding on the conversation with Roland and Isabella, I plotted residuals of the LME across sessions but I am not seeing a huge difference in variance. I also plotted residuals vs fitted. I had 2 versions of LME: model1 with the term (1 | subject) and model2 with the term (sessionIndex | subject). Model1 shows non-constant variance in this plot and model2 shows constant variance.

Does this mean LME using model2 is good and that's why it has a lower MSE? I am not really understanding it because my reference says LME is for normally distributed response variables. The data I am using does not have a normally distributed response variable so I would expect GEE to be better than LME. Any thoughts?