53
$\begingroup$

Given all good properties of state-space models and KF, I wonder - what are disadvantages of state-space modelling and using Kalman Filter (or EKF, UKF or particle filter) for estimation? Over let's say conventional methodologies like ARIMA, VAR or ad-hoc/heuristic methods.

Are they hard to calibrate? Are they complicated and hard to see how a change in a model's structure will affect predictions?

Or, put another way - what are advantages of conventional ARIMA, VAR over state-space models?

I can think only of advantages of a state-space model:

  1. It easily handles structural breaks, shifts, time-varying parameters of some static model - just make those parameters dynamic states of a state-space model and model will automatically adjust to any shifts in parameters;
  2. It handles missing data very naturally, just do transition step of KF and don't do update step;
  3. It allows to change on-a-fly parameters of a state-space model itself (covariances of noises and transition/observation matrices) so if your current observation came from a little different source than others - you can easily incorporate it into estimation without doing anything special;
  4. Using above properties it allows easily handle irregular-spaced data: either change a model each time according to interval between observations or use regular interval and treat intervals without observations as missing data;
  5. It allows to use data from different sources simultaneously in the same model to estimate one underlying quantity;
  6. It allows to construct a model from several interpretable unobservable dynamic components and estimate them;
  7. Any ARIMA model can be represented in a state-space form, but only simple state-space models can be represented exactly in ARIMA form.
$\endgroup$
  • 1
    $\begingroup$ JFew additional advantages, as noted in your first point, can easily incorporate multiple level shifts and outliers. In my experience structural breaks can be easily identified with state space than ARIMA. Also can easily incorporate nonlinear effects of exogenous variables. Does not require the time series data to be stationary which is a huge +. $\endgroup$ – forecaster Dec 3 '13 at 1:23
  • 2
    $\begingroup$ Well I will take state space models over ARIMA any time. I can think of two disadvantages (sort of): a) corresponding state space model of an ARIMA model has a lot of unnecessary zeros in the design matrices. One may argue that ARIMA is more compact. b) there are non-linear/non-gaussian models which rarely have analytical forms that can be sometimes described in ARIMA like form, but will be difficult in traditional state-space. $\endgroup$ – Cagdas Ozgenc Dec 4 '13 at 17:19
  • 2
    $\begingroup$ @Kochede Durbin and Koopman could not seem to think of many disadvantages either -- they mentioned two on the bottom of page 52 in their fantastic textbook. And I'd say these disadvantages are not necessarily true anymore. $\endgroup$ – user5594 Dec 6 '13 at 18:01
  • 2
    $\begingroup$ A few questions . Does it clearly identify time trend changes and report the points in time where the trend changes ? Does it distinguish between parameter changes and error variance changes and report on this ? Does it detect and report on specific lead and lag effects around user specified predictors ? Can one specify the minimum number of values in a group before a level shift/local time trend is declared? Does it distinguish between the need for power transforms versus deterministic points in time where the error variance changes ? ARMAX models speak to all of these considerations. $\endgroup$ – IrishStat Dec 7 '13 at 12:03
  • 1
    $\begingroup$ For completeness, a disadvantage in some circumstances is that you have to explain them. That depends on your audience. I am happy if anyone wants to dismiss this as something that is, or should be, irrelevant to choosing a technique. $\endgroup$ – Nick Cox Dec 12 '13 at 10:24
25
$\begingroup$

Here is some preliminary list of disadvantages I was able to extract from your comments. Criticism and additions are very welcome!

Overall - compared to ARIMA, state-space models allow you to model more complex processes, have interpretable structure and easily handle data irregularities; but for this you pay with increased complexity of a model, harder calibration, less community knowledge.

  1. ARIMA is a universal approximator - you don't care what is the true model behind your data and you use universal ARIMA diagnostic and fitting tools to approximate this model. It is like a polynomial curve fitting - you don't care what is the true function, you always can approximate it with a polynomial of some degree.
  2. State-space models naturally require you to write-down some reasonable model for your process (which is good - you use your prior knowledge of your process to improve estimates). Of course, if you don't have any idea of your process, you always can use some universal state-space model also - e.g. represent ARIMA in a state-space form. But then ARIMA in its original form has more parsimonious formulation - without introducing unnecessary hidden states.
  3. Because there is such a great variety of state-space models formulations (much richer than class of ARIMA models), behavior of all these potential models is not well studied and if the model you formulated is complicated - it's hard to say how it will behave under different circumstances. Of course, if your state-space model is simple or composed of interpretable components, there is no such problem. But ARIMA is always the same well studied ARIMA so it should be easier to anticipate its behavior even if you use it to approximate some complex process.
  4. Because state-space allows you directly and exactly model complex/nonlinear models, then for these complex/nonlinear models you may have problems with stability of filtering/prediction (EKF/UKF divergence, particle filter degradation). You may also have problems with calibrating complicated-model's parameters - it's a computationally-hard optimization problem. ARIMA is simple, has less parameters (1 noise source instead of 2 noise sources, no hidden variables) so its calibration is simpler.
  5. For state-space there is less community knowledge and software in statistical community than for ARIMA.
$\endgroup$
  • 1
    $\begingroup$ Are you aware of any real-live example/industrial applications in which a Kalman filter performs better than a simple moving average or exp smoothing in forecasting a time series provided there is no clear underlying model (so exclude models arising from the laws of physics)? In most papers the performance looks very similar (and academic papers have a positive performance bias for new, original, complex models). In most of the cases there is no knowledge of a proper linear state system model and covariances, etc, required to specify a Klaman filter... $\endgroup$ – Mannaggia May 2 '14 at 7:59
  • $\begingroup$ This is true. Still I have an example in my practice. When you have some general purpose model (like linear regression) then you can make its parameters states of Kalman Filter and estimate them dynamically. Of course, you can also just refit your model at each time-step, but this is much more computationally expensive than a single KF update. If in reality parameters indeed vary over time or if you model is not match exactly the real process - this may help to fit your model better and improve its performance. $\endgroup$ – Kochede Oct 23 '14 at 3:31
  • 1
    $\begingroup$ I can't find any references for ARIMA being a universal universal approximator other than your post. Could you point me to one? $\endgroup$ – Reinstate Monica Jan 21 '18 at 18:48
  • 2
    $\begingroup$ @Alex This follows from the Wold's decomposition theorem, for example see here phdeconomics.sssup.it/documents/Lesson11.pdf $\endgroup$ – Kochede Jan 23 '18 at 7:30
  • 1
    $\begingroup$ Can I say state space model has more general form and ARIMA only covers a subset of it? $\endgroup$ – Vickyyy Jun 5 at 17:11
3
$\begingroup$

Thanks @IrishStat for several very good questions in comments, the answer for your questions is too long to post as comment, so I post it as an answer (unfortunately, not to original question of the topic).

Questions were: "Does it clearly identify time trend changes and report the points in time where the trend changes ? Does it distinguish between parameter changes and error variance changes and report on this ? Does it detect and report on specific lead and lag effects around user specified predictors ? Can one specify the minimum number of values in a group before a level shift/local time trend is declared? Does it distinguish between the need for power transforms versus deterministic points in time where the error variance changes?"

  1. Identify trend changes - yes, most naturally, you can make trend-slope one of state-variables and KF will continuously estimate current slope. You can then decide what slope-change is big enough for you. Alternatively, if slope is not time-varying in your state-space model, you can test residuals during filtering in a standard way to see when there is some break of your model.
  2. Distinguish between parameters changes and error variance changes - yes, variance can be one of parameters(states), then which parameter most likely changed depends on a likelihood of your model and how particularly data have changed.
  3. Detect lead/lag relations - not sure about this, you certainly can include any lagged vars into a state-space model; for selection of lags, you can either test residuals of models with different lags included or, in a simple case, just use a cross-correlogram before formulating a model.
  4. Specify threshold number of observations to decide trend change - yes, as in 1) because filtering is done recursively, you can not only threshold slope change that is big enough for you, but also # of observations for confidence. But better - KF produces not only estimate of slope, but also confidence bands for this estimate, so you may decide that slope changed significantly when its confidence bound passed some threshold.
  5. Distinguish between need for power-transform and need for bigger variance - not sure I understand correct, but I think you can test residuals during filtering to see if they are still normal with just bigger variance or they got some skew so that you need to change your model. Better - you may make it a binary switching state of your model, then KF will estimate it automatically based on likelihood. In this case model will be non-linear so you will need UKF to do filtering.
$\endgroup$
1
$\begingroup$

The Kalman Filter is the optimal linear quadratic estimator when the state dynamics and measurement errors follow the so-called linear Gaussian assumptions (http://wp.me/p491t5-PS). So, as long as you know your dynamics and measurement models and they follow the linear Gaussian assumptions, there is no better estimator in the class of linear quadratic estimators. However, the most common reasoners for "failed" Kalman Filter applications are:

  1. Imprecise/incorrect knowledge of the state dynamics and measurement models.

  2. Inaccurate initialization of the filter (providing an initial state estimate and covariance that is inconsistent with the true system state). This is easily overcome using a Weighted Least Squares (WLS) initialization procedure.

  3. Incorporating measurements that are statistical "outliers" with respect to the system dynamics model. This can cause the Kalman Gain to have negative elements, which can lead to a non positive semi-definite covariance matrix after update. This can be avoided using "gating" algorithms, such as ellipsoidal gating, to validate the measurement prior to updating the Kalman Filter with that measurement.

These are some of the most common mistakes/issues I've seen working with the Kalman Filter. Otherwise, if the assumptions of your models are valid, the Kalman Filter is an optimal estimator.

$\endgroup$
1
$\begingroup$

You can refer to the excellent book Bayesian forecasting and dynamic models (Harrison and West, 1997). The authors show that almost all traditional time series models are particular cases of the general dynamic model. They also emphasize the advantages. Perhaps one of the major advantages is the easiness with which you can integrate many state space models by simply augmenting the state vector. You can, for example, seamlessly integrate regressors, seasonal factors, and an autoregressive component in a single model.

$\endgroup$
  • $\begingroup$ Hi, can you elaborate more on "You can, for example, seamlessly integrate regressors, seasonal factors, and an autoregressive component in a single model."? Correct me if I am wrong, does that mean for ARIMA you need to make the time series stationary but for state space model you only need to tweak the number of state variables? $\endgroup$ – Vickyyy Jun 5 at 17:13
  • $\begingroup$ @Vickyyy Unlike ARIMA, state space models do not assume stationarity. You can just add many components to it and represent them in a single state vector. $\endgroup$ – Anselmo Jun 5 at 18:32
1
$\begingroup$

I'd add that if you directly use a State Space function, you're probably going to have to understand the several matrices that make up a model, and how they interact and work. It's much more like defining a program than defining an ARIMA model. If you're working with a dynamic State Space model, it gets even more complicated.

If you use a software package that has a really, really nice State Space function, you may be able to avoid some of this, but the vast majority of such functions in R packages require you to jump into the details at some point.

In my opinion, it's a lot like Bayesian statistics in general, the machinery of which takes more understanding, care, and feeding to use than more frequentist functions.

In both cases, it's well worth the additional details/knowledge, but it could be a barrier to adoption.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.