I'm working with fish migration time series data, and I am modeling fish counts using environmental variables. I am using a generalized additive mixed model from the mgcv package in R. I am using a negative binomial distribution and year as a random effect. Additionally, the data has auto-correlation, and I have tried incorporating different auto-correlation structures with day nested within year, including AR1 and ARMA.
gamm(FishCount ~ s(Var1) + s(Var2) + Var3 + Var4, family = negbin(0.82), random = list(Year = ~ 1), correlation = corARMA(form = ~ Day | 1, p = 1, q = 1))
Including the correlation structure gives error free output that improves the auto-correlation, substantially drops the AIC, and also improves the residual plots. However, I have not seen a correlation structure included in a negative binomial model before.
Is it valid to include AR1 or ARMA correlation structures into a negative binomial GAMM?