I am computing a very simple Kalman filter (random walk + noise model).
I find that the output of the filter is very similar to a moving average.
Is there an equivalence between the two?
If not, what is the difference?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ Not an answer, but you could probably calculate the kalman filter steps analytically for this simple model, as it would only involve small matrices. And which "Kalman Filter" value are you comparing: the smoothed value, 1-step ahead prediction,..? $\endgroup$– probabilityislogicOct 4, 2011 at 12:35
-
$\begingroup$ just the filter of the kalman filter: $\theta_t|y_t$ $\endgroup$– RockScienceOct 5, 2011 at 2:58
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
5
A random walk + noise model can be shown to be equivalent to a EWMA (exponentially weighted moving average). The kalman gain ends up being the same as the EWMA weighting.
This is shown to some details in Time Series Analysis by State Space, if you Google Kalman Filter and EWMA you will find a number of resources that discuss the equivalence.
In fact you can use the state space equivalence to build confidence intervals for EWMA estimates, etc.
-
1$\begingroup$ so apart for the confidence interval, what is the point of adding complexity with the state space models? EWMA seems much simpler to understand, implement, manipulate $\endgroup$ Oct 5, 2011 at 3:00
-
1$\begingroup$ The equivalence holds only for certain models, e.g. random walk + noise ~ EWMA or local linear trend ~ holt-winters EWMA. State space models are a lot more general than custom smoothers. Also initialization has sounder theoretical bases. If you want to stick to random walk + noise, and you are not familiar with the Kalman filter, then you might be better off with EWMAs. $\endgroup$– Dr GOct 5, 2011 at 8:01
-
$\begingroup$ Thank you for you explanation, I understand DLMs are more general than classic smoothers. In your experience, does the complexity of state space models adds value? $\endgroup$ Oct 5, 2011 at 9:14
-
$\begingroup$ Difficult to say, if you can spare the time I'd argue state space models can be a useful technique to learn. $\endgroup$– Dr GOct 5, 2011 at 10:03
-
$\begingroup$ at least your answer shows that kalman filter adds value only if the model is more complex than EWMA. $\endgroup$ Mar 19, 2013 at 12:54
$\begingroup$
$\endgroup$
3
To Start: The equivalence of Kalman filter with EWMA is only for the case of a "random walk plus noise" and it is covered in the book, Forecast Structural Time Series Model and Kalman Filter by Andrew Harvey. The equivalence of EWMA with Kalman filter for random walk with noise is covered on page 175 of the text. There the author also mentions that the equivalence of the two was first shown in 1960 and gives the reference to it. Here is the link for that page of the text: https://books.google.com/books?id=Kc6tnRHBwLcC&pg=PA175&lpg=PA175&dq=ewma+and+kalman+for+random+walk+with+noise&source=bl&ots=I3VOQsYZOC&sig=RdUCwgFE1s7zrPFylF3e3HxIUNY&hl=en&sa=X&ved=0ahUKEwiK5t2J84HMAhWINSYKHcmyAXkQ6AEINDAD#v=onepage&q=ewma%20and%20kalman%20for%20random%20walk%20with%20noise&f=false
Now here is reference which covers an ALETERNATIVE to the Kalman and Extended Kalman filters -- it yielded results that match the Kalman filter but the results are obtained much faster! It is "Double Exponential Smoothing: An Alternative to Kalman Filter-Based Predictive Tracking." In Abstract of the paper (see below) the authors state "...empirical results supporting the validity of our claims that these predictors are faster, easier to implement, and perform equivalently to the Kalman and extended Kalman filtering predictors..."
http://cs.brown.edu/~jjl/pubs/kfvsexp_final_laviola.pdf
This is their Abstract "We present novel algorithms for predictive tracking of user position and orientation based on double exponential smoothing. These algorithms, when compared against Kalman and extended Kalman filter-based predictors with derivative free measurement models, run approximately 135 times faster with equivalent prediction performance and simpler implementations. This paper describes these algorithms in detail along with the Kalman and extended Kalman Filter predictors tested against. In addition, we describe the details of a predictor experiment and present empirical results supporting the validity of our claims that these predictors are faster, easier to implement, and perform equivalently to the Kalman and extended Kalman filtering predictors."
-
1$\begingroup$ I'm don't think this really answers the question about why the Kalman filter and MA give similar results, but it is tangentially related. Could you add a full reverence for the paper you cite, rather than a bare hyperlink? This would future-proof your answer in case the external link changes. $\endgroup$ Apr 8, 2016 at 5:46
-
$\begingroup$ It wasn't suppose be. Like the introduction says, it's meant to be an alternative to Kalaman but much faster. If it or another method was "exactly" the same as Kalman, based on the topic of the article, the author would have mentioned it. So in that respect the question is answered. $\endgroup$– jimmehApr 9, 2016 at 12:15
-
$\begingroup$ The equivalence of Kalman filter to random walk with EWMA is covered in the book Forecast Structural Time Series Model and Kalman Filter by Andrew Harvey. The equivalence of EWMA with Kalman filter for random walk is covered on page 175 of the text. There he mentions that it was first shown in 1960 and gives the reference. $\endgroup$– jimmehApr 9, 2016 at 12:54