# Specify correlation structure for different groups in mixed-effects model (lme4/nlme)

I am trying to account for spatial autocorrelation in a linear mixed-effects model in R with measurements repeated in time. BodyMass has been collected once per Year in 150 different Sites over a 4-year period. Temporal autocorrelation should be negligible as body mass measurements are taken from dead animals.

As correlation structures of the form correlation= are not accepted in the lmer() model in lme4, I tried lme() from the older nlme package instead.

My baseline model looks like this:

library(nlme)
M1 <- lme(BodyMass~Year, data=df, random=~1|Site)


Now I want to account for spatial autocorrelation, considering the grouped spatial locations (each combination of coordinates is repeated 4 times due to the 4-year period). This syntax does however not work:

M2 <- lme(BodyMass~Year, data=df, random=~1|Site, correlation=corAR1(form=~x+y|Year))


It gives the following error:

Error in lme.formula(BodyMass ~ 1, data = df, random = ~1 | Site,  :
incompatible formulas for groups in 'random' and 'correlation'


According to Ben Bolker's answer here, the reason for that error is that

lme() insists that the grouping variable for the random effects and for the correlation be the same.

In my model, it would however not make sense to use the same grouping variable, as years are grouping the correlation structure (by the repeated coordinates), while years are inappropriate as a random factor, as they are consisting only of 4 levels (as discussed here).

Using gls() instead, as suggested here, is no real option for me as I need to account for the random factor Site and like to use R-squared, which is only possible in ordinary least squares (OLS) methods.

(1) Do you know any way on how to account for spatial autocorrelation in this case, keeping my random factor as well as the OLS framework to be able to calculate R-squared values?

EDIT: AdamO wrote in his answer to the differences between lme4 and nlme (link):

The "big three" correlation structures of: Independent, Exchangeable, and AR-1 correlation structures are easy handled by both packages.

(2) How is it possible to include these "big three" correlation structures in lme4?

I'm thinking that it might be possible to circumvent the grouping variable problem of nlme by using lme4 instead, if it's actually possible to handle spatial autocorrelation in there.

EDIT 2: I found that changing the grouping variable for the correlation structure as follows makes the model work without an error:

M3 <- lme(BodyMass~Year, data=df, random=~1|Site, correlation=corAR1(form=~x+y|Site/Year))


(3) Does this expression for the grouping variable in the correlation structure make sense?

I'm not entirely sure if it does, because in a a random factor, |Site/Year would imply that year is nested within site, but I think year and site would rather be crossed as each site is represented in every year.