Kalman Filter vs. Regression

Question

I'm an economics undergraduate with a fundamental understanding of regression and some experience with machine learning models (e.g. regression trees, boosting). To my knowledge, Kalman Filter is superior in that 1. it can converge to a reliable estimate quickly without the entire population data, and 2. as it updating based on the errors of both the prior estimate and the measurement, it is computationally faster than say rerunning an entire regression.

My application is intended for the finance industry (e.g. asset prices & returns data). My question is: is Kalman Filter more accurate? I think that as a state-space model, it is able to adapt to time series with irregular intervals. As an online model, it should also be far more performant than say a traditional regression model. I think that it is also more accurate than a traditional ARMA model, but I'm not sure how?

Also, I think I have some fundamental misconceptions about the Kalman Filter. From my understanding, it is an algorithm and not a model—what does that mean? K-means is an algorithm that separates data into homogeneous clusters; heterogeneously apart. This answer refers to Kalman Filter as a linear estimator—does that mean that KF is just a way of estimating things, and isn't actually a different way of modelling things?

Answer 1

Building a state space model involves defining two basic distributions:

1. The transition distribution of the state: $p(X_t|X_{t-1}, \theta)$.
2. The observation distribution, conditional on the state: $p(Y_t|X_t, \theta)$

In the special case where the state space model is linear and Gaussian, the Kalman filter is then a way of deriving the associated sequence of filtering distributions $p(X_t|Y_1, ..., Y_t, \theta)$, for each $t$, recursively. It also produces the likelihood, as a function of the parameters $\theta$, allowing for maximum likelihood estimation of those parameters. So, the state space model is the model, and the Kalman filter is an algorithm which allows for estimation of that model, yes.

I'm not sure what you are getting at with the Kalman filter being "superior" to regression, but you can consider the Kalman filter to be a generalization of least squares: there is a state space model that corresponds to running a regression, and the mean of the last filtering distribution is exactly the least squares estimate. If you do that, and you save the necessary information about the state of the filter, then yes you can add one more observation and compute the least squares estimate by running just one more step of Kalman filter, without refitting the whole thing. Of course, if that basic linear regression model is wrong (say, the coefficients should vary over time rather than be fixed), then you can handle a wider class of models with the Kalman filter so, sure, in that sense it is "superior".

As for ARMA: every ARMA model is a linear Gaussian state space model, and in fact most software implementations for the estimation of ARMA models begin by casting them to state space form and then use the Kalman filter. If by "more accurate" you mean "better at prediction", then in some sense because the state space class is strictly larger than ARMA, you can fit non-ARMA state space models with the Kalman filter which may provide better forecasts (although that is certainly not guaranteed in practice).