# Kalman filter vs Kalman Smoother for beta calculations

I am trying to calculate the beta of two timeseries by setting up a state-space model, calculating its covariances via the EM algorithm and finally running the kalman filter/smoother. From what I have read, I understand that using the kalman smoother might more sense for what I am trying to achieve since I am in a post-processing environment and not really looking for any real-time predictions.

However, when comparing the two approaches with a a few examples, there are cases where the smoother does seem to improve the beta numbers whereby it feels similar to removing the noise of the kalman filter values via a spline interpolation (see the first picture below) and other cases where the 'interpolation' is quite broad and seems to be 'hiding' a few of the underlying dynamics of the kalman filter results (please see the second picture for example)

Any thoughts on which is the best approach?

EDIT:

adding the difference between the values of the smoother and the filter, as expected the difference goes towards zero in the most recent observations and is actually zero in the latest one:

... -0.03943203, -0.01329412, -0.011849 , -0.01031422, -0.01596532, -0.01822451, -0.00513093, 0.00208434, 0.00244347, -0.0020279 , -0.00991458, -0.0046458 , 0. ])

• What is "the beta between time-series"? (I guess this relates e.g. beta in CAPM, but unclear). – Juho Kokkala Oct 18 '17 at 18:02

They are not really different approaches in that they are solutions to different problems: one computes the sequence of filtering distributions $p(\beta_t|Y_{1:t})$, and the other the distributions based on all observations $p(\beta_t|Y_{1:T})$, for $t =1,...,T$.

The smoother doesn't "hide underlying dynamics" but rather adjusts its state estimate (with respect to the filter) to reflect the fact that new data has been observed; what "looked like" an increase in $\beta$ at time $t$ is now, on the basis of more accumulated evidence, believed to have been mostly observation noise and a much smaller move in $\beta$.

Which algorithm you should use depends on what you need it for. If you are really looking at this data purely retrospectively as you mention, then smoother is what you want. If you want to build a trading algorithm based on $\beta_t$ then you obviously need to use the filtered estimate in your backtesting because the smoothed one will not be available when you actually use the strategy (it depends on future data).