GAM using a cyclic spline improves residual structure but reduces fit I'm working on a dataset monitoring soil moisture levels throughout the summer. The general trend in the data is the following:

When I use a GAM with default thin-plate spline and AR(1)process there is still a clear residual pattern:

Now if instead I use a cyclic spline the residual pattern improves, but observed vs. fitted values get's a bit worse, this model also has a higher AIC value:

My Question is whether it is appropriate to go for the cyclic spline here as it improves the residual structure, even though overall model fit is worse.
 A: The fast answer is NO, you should not use periodic splines. In a way you are trying to fix a secondary problem (heteroskedasticity) by unfixing the primary problem, to get the predictions right. 
Some more details:  Your first plot shows lower variance in the early summer part of the plot, higher variance for later summer. In this case, that is lower variance goes with generally higher values. Transformations are mostly used for the opposite pattern, so are probably not of much use here. In addition, they will complicate the modeling, so stay away from transformations. 
Your second&third plot (model with smoothing splines) is what you would expect in this case: more variability among residuals when the variance in the data is higher. That is just a part of the phenomenon to be modelled, not really a problem. The heteroskedasticity is not strong enough to warrant much, one could use weighted estimation, but you would estimate the weights from the data, probably not gaining much, and introducing further problems (how do you get correct standard errors when the weights are estimated? You could try bootstrapping, my guess is that you would not gain anything ...)
The last plots (model with periodic smoothing splines) confirms this: Yes, now the residuals have about constant variance, but it is everywhere higher! So you have gained nothing. Maybe try robust standard errors.  If you are using R there is some useful hints and code here: How to calculate the robust standard error of predicted y from a linear regression model in R?  (which is closed for unfathomable reasons, it seems understandable and useful)
