How to do model selection in dynamic linear model?

I am trying to use DLM to model a time series. Candiate model includes local level, local trend and local trend with seasonal part. I do not know how to do model selection. Can AIC be calculated? I found no function in the R package dlm.

• I assumed you were talking about R package; correct me if I'm wrong.
– user88
Mar 21, 2011 at 13:50

The dlmMLE function in the dlm package will compute the likelihood. Then

AIC = -2 log(likelihood) + 2p

where p is the number of parameters estimated. You might like to read the vignette for the dlm package which contains a lot of helpful information and examples.

However, a much simpler approach is to use the StructTS function in the stats package (which is automatically loaded). It will fit the model you want, and returns the loglikelihood.

I do not know the answer, so I can only offer some thoughts hoping that someone else can throw light on the issue.

It seems to me that there is no problem in computing the likelihood. To compute the value of the AIC criterion (which may or may not make sense in this context), what would be required is the number of fitted parameters.

Here is were things become slippery. To take the simplest example, the (univariate) local level model requires fitting two parameters (the variances of the state and of the noise), but those are really best described as metaparameters. If the variance of the state is zero, you are fitting a single mean (1 parameter). If the variance of the state goes to infinity, you are effectively fitting one parameter per observation.

One way out is to define the "number of equivalent parameters" as in,

Hodges, J. S. and Sargent, D. J. (2001) Counting Degrees of Freedom in Hierarchical and Other Richly-Parameterised Models, Biometrika, 88(2), p. 367-379.

and this is what I have done in my own work (on purely heuristic grounds and with some trepidation!). See for instance,

There are alternative ways of calculating "equivalent parameters"; if you want to follow this route I can give you some references.

• The "equivalent parameters" approach sounds like what I've been wondering about. I'll check the references tonight. Also, I have a DLM which uses a local level model to calculate the trend, with five points where I increase the variance (in W) in order to handle level shifts. Is this two parameters (the baseline W and the level-shift W), or do I need to take into account that there are five level shifts? Perhaps no AIC-like approach would account for things like identified level shifts, outliers, etc. Oct 25, 2011 at 15:09
• My feeling is that this is MORE than two parameters, as you are also selecting the level-shift point. How many, I do not know. Perhaps the MDL literature may throw some light (you need to use some bits to say where the shift level takes place). Not sure. Tell me if you have some insight. Oct 26, 2011 at 7:37

From wiki "Given a data set, several candidate models may be ranked according to their AIC values. From the AIC values one may also infer that e.g. the top two models are roughly in a tie and the rest are far worse. Thus, AIC provides a means for comparison among models—a tool for model selection. AIC does not provide a test of a model in the usual sense of testing a null hypothesis; i.e. AIC can tell nothing about how well a model fits the data in an absolute sense. Ergo, if all the candidate models fit poorly, AIC will not give any warning of that."

Thus the AIC should bot be used to do model selection. After reading up on DLM it appears to me that this approach is MODEL-BASED rather than DATA-BASED. One is assuming a model when using DLM. A DLM model uses a mixture of high order polynomials in time and fixed trigonometric structures which might be useful for certain physical time series. This approach does not lend itself to generating a set of residuals free of structure which meet the Gaussian requirements. A useful alternative is to form the model empirically detecting as needed any auto-projective structure, any deterministic structure (Pulses, level Shifts, Seasonal Pulses and.or Local Time Trends via INTERVENTION DETECTION also referred to as OUTLIER DETECTION.