What defines an observation in AIC? I have a question based upon a section of this paper: "Model selection for dynamical systems via sparse regression and information criteria", which can be found here: https://arxiv.org/abs/1701.01773
It states the following:
My question is: How can the sum over each time series residual be called average error (no division) and how can that be used in place of the likelihood function? How is this mathematically sound?
 A: @Oarz There is a typo in the $E_{avg}$ equation. It should be divided by the number of points in the time series so that it is the average error at each time-point for that time series. This is correct in the GitHub implementation of the code available here: https://github.com/niallmm/SINDy_AIC/%E2%80%8B
As for your main question, we are treating each time series as an observation, rather than each point in the time series. We take many initial conditions. For these synthetic examples, the observation error at each time point is drawn from a Gaussian distribution. So for the correct model, the residuals at each point are Gaussian and the average error on each time series is also Gaussian. The only assumption needed to substitute into AIC for the likelihood function is that the error is Gaussian. In practice, the error for the incorrect models will not be Gaussian, but it will be "worse" than Gaussian so the AIC will still have the correct trend. I found (through numerical experiment) that taking each time point as an observation did not work well but do not have a rigorous explanation as to why this happens.
Usually, AIC is used on in-sample data (ie the same as the training data), not out f sample data. For most of the examples shown most of the AIC value is coming from the effect of out-of-sample cross-validation, not the AIC penalty. However, for a small number of time series (observations) the penalty does matter.
It may be that other methods such as minimum descriptive length would be more mathematically rigorous for use on time-series, but I have not performed the comparison of actual performance.
