I trained two basic feed-forward neural networks on time series data. The first one uses the observation at time step $t$ to predict $t+1$. Hence, it only has one predictor variable. The second network uses a temporal lag of size 1, i.e. it uses the observation at time step $t$ and $t-1$ to predict $t+1$. Hence, it uses two predictor variables.
Comparing the MSE of both models reveals, as expected, that the second network (the one that with the temporal lag) predicts better. However, the first model yields the lower AIC, probably because it has less parameters (I calculated the likelihood function of the models using the number of samples and the MSE).
If I compare the 10-fold cross-validated (CV) MSE of the models instead of their AICs, the second model is preferred, despite its larger number of parameters.
So, which model should I choose? AIC says the first model, CV MSE the second one.