I am trying to calculate the beta of two timeseries by setting up a state-space model, calculating its covariances via the EM algorithm and finally running the kalman filter/smoother. From what I have read, I understand that using the kalman smoother might more sense for what I am trying to achieve since I am in a post-processing environment and not really looking for any real-time predictions.
However, when comparing the two approaches with a a few examples, there are cases where the smoother does seem to improve the beta numbers whereby it feels similar to removing the noise of the kalman filter values via a spline interpolation (see the first picture below) and other cases where the 'interpolation' is quite broad and seems to be 'hiding' a few of the underlying dynamics of the kalman filter results (please see the second picture for example)
Any thoughts on which is the best approach?
adding the difference between the values of the smoother and the filter, as expected the difference goes towards zero in the most recent observations and is actually zero in the latest one:
... -0.03943203, -0.01329412, -0.011849 , -0.01031422, -0.01596532, -0.01822451, -0.00513093, 0.00208434, 0.00244347, -0.0020279 , -0.00991458, -0.0046458 , 0. ])