Confidence interval of Kalman Filter vs. OLS on expanding window (RLS)

Question

I am estimating a regression model with time-varying parameters with Kalman filter assuming that the coefficients follow a random walk. Then, I use OLS on expanding window, a.k.a. recursive least squares (RLS) to compare the results. I find that the confidence intervals around the KF estimates are much wider than those around the RLS estimates. My intuition for this is that, in each step of the filtering recursion, we add the state disturbances variance-covariance matrix to the predicted variance-covariance matrix of the states. To make it concrete, take the state-space model:

$y_t = x_t\beta_t +\epsilon_t$ where $\epsilon_t\sim N(0,\Sigma_{\epsilon})$ (1) i the measurement equation

$\beta_{t+1}=\beta_{t}+\eta_{t}$ where $\eta_t\sim N(0,\Sigma_{\eta})$ (2) is the state equation

where $Y_t = [y_1 y_2 ... y_t]'$ is $T\times 1$ column vector, $\beta_t$ and $\eta_t$ are $m\times 1$ column vectors, $x_t$ is $1*m$ part of $X_t=[x_1' x_2' ... x_t']'$ which is $t\times m$, $\epsilon_t$ and $y_t$ are $1\times 1$, $\Sigma_{\epsilon}$ is $1*1$ and $\Sigma_{\eta}$ is $m\times m$.

The KF recusion reads:

$v_t = y_t - x_t\beta_t$ (3) is the prediction error

$F_t = x_tP_tx_t'+ \Sigma_{\epsilon}$ (4) is the prediction error variance

$\beta_{t|t} = \beta_t + P_tx_t'F_t^{-1}v_t$ (5) the filtered state

$P_{t|t} = P_t - P_tx_t'F_t^{-1}x_tP_t$ (6) the filtered state variance

$\beta_{t+1} = \beta_{t} + K_tv_t = \beta_{t|t}$ (7) predicted state

$P_{t+1} = P_t(1-K_tx_t)' + \Sigma_{\eta} = P_{t|t} + \Sigma_{\eta}$ (8) predicted state variance

where $K_t = P_tx_t'F_t^{-1}$ is the Kalman Gain,

$\Sigma_{\eta}$ and $\Sigma_{\epsilon}$ are estimated via ML and diffuse initialisation is used with $\beta_1 = 0$ and $P_1 = 10^7I_{m*m}$.

Recursive least squares(RLS) is obtained if $\Sigma_{\eta}=0$. In that case, (5) equals (7) and (6) equals (8), so that filtered and predicted states and their variances are the same. Of course, filtered and predicted were already the same before (because we assumed a random walk). If we estimate OLS using the sample from period 1 until $m+1$ and if we use those OLS estimates, say $\hat{\beta}_{m+1}$ as initial state $\beta_1$ and $X_{m+1}'X_{m+1})^{-1}$ as $P_1$, we get that the KF reduces to RLS.

This is because RLS can be written as:

$\hat{\beta}_t=\hat{\beta}_{t-1}+\frac{(X_{t-1}'X_{t-1})^{-1}x_t'(y_t-x_t\hat{\beta}_{t-1})}{x_t(X_{t-1}'X_{t-1})^{-1}x_t'+1}$

and

$(X_t'X_t)^{-1} = (X_{t-1}'X_{t-1})^{-1} - \frac{(X_{t-1}'X_{t-1})^{-1}x_t'x_t(X_{t-1}'X_{t-1})^{-1}}{x_t(X_{t-1}'X_{t-1})^{-1}x_t'+1}$

for $t= m+2,m+3,...T$

Clearly, the KF gives us $P_{t|t}=P_{t+1}=(X_{t}'X_{t})^{-1}$ and $\beta_{t|t}= \hat{\beta}_t$.

Now, my question is why, the RLS algorithm (or the KF with time-invariant coefficients if you prefer), gives me, quite often, narrower confidence intervals than the KF with time-varying coefficients. It is clear thatin (8) we keep on adding $\Sigma_{\eta}$ which we do not do under RLS; however, we estimate $\Sigma_{\eta}$ via ML (so it could be close to 0) and that noise is used by the Kalman Gain to optimally weight $\beta_t$ and $v_t$ in (5) and (7). My only explanation is that the true DGP has time-invariant coefficients or that the random walk is a bad model for the state equation and therefore the Kalman Gain is not able to use the noise that $\Sigma_{\eta}$ creates to get more accurate estimates of the state. However, I am not totally convinced by this.

In other words, if the KF is the minimum variance unbiased linear estimator, since the RLS is a special case of the KF, it cannot be minimum variance unless it is biased or not linear, which it is not since it is OLS, or unless RLS coincides with KF; if the RLS is minimum variance, then the KF should estimate exactly the same as RLS; whenever they diverge, KF should beat RLS.

extra info to reply to hejseb:

recursive least squares: ols in an expanding window, basically. in this paper https://link.springer.com/article/10.1007/BF01973191 , equations (4) and (5) define the RLS algorithm (which is equivalent to running one OLS regression from time 1 till t and getting coefficients for time t, then from 1 till t+1 and getting coefficients for time t+1, and so on until you run an OLS regression over the full sample from 1 until T).

extra info to reply to Taylor:

In the case of the RLS, the confidence intervals are simply $\hat{\beta}_t +- 1.645*sqrt(diag(\sigma^2(X_t´X_t)^{-1}))$ where $\sigma^2=(T-m)^{-1}(\epsilon'\epsilon)$. In the case of the KF they are $\beta_{t|t}+-1.645*sqrt(diag(P_{t|t}))$, where 1.645 is the critical value I chose. Of course, we have that $diag(P_{t|t}) >diag(\sigma^2(X´_tX_t)^{-1})$. My question is why. I would expect the opposite. Note that $\sigma^2$ is time-varying if I apply the OLS directly (I get a different $\sigma$ estimate each time I run OLS over the expanding window; however, if I use the KF with time-invariant coefficients, I can estimate it as $(T-m)^{-1}\sum(v_t^2/F_t)$ and make it time-invariant; if I use the RLS, I can just compute the residuals at the end of the recursion and use $\sigma^2=(T-m)^{-1}\epsilon'\epsilon$. This does not lead to any differences so this is not the reason. Note that in the KF with time-varying coefficients, $\Sigma_{\epsilon}$ is time-invariant and estimated via ML.

Answer 1

Here are a few notes. They don't answer your question succinctly, but they might be helpful. To be honest, I had a hard time following a lot of what you were saying, so at the very least, this could help clarify the question.

Some notation:

Let model 1 be: $$ y_t = x_t\beta_t +\epsilon_t \\ \beta_{t+1}=\beta_{t}+\eta_{t} $$ with $\epsilon_t\sim N(0,\Sigma_{\epsilon})$ and $\eta_t\sim N(0,\Sigma_{\eta})$.

Let model 2 be: $$ y_t = x_t\beta_t +\epsilon_t \\ \beta_{t+1}=\beta_{t}. $$ with $\epsilon_t\sim N(0,\Sigma_{\epsilon})$ (same covariance matrix as model 1).

Note 1

Assuming your formulas are correct, and assuming both models share the observation covariance matrix, plugging (8) into (6) gives you some recursive formulas for your filter covariance. For the first model you get $$ P^1_{t|t} = f_{1,t}(P_{t-1|t-1}) $$ say, and for the second model you get $$ P^2_{t|t} = f_{2,t}(P^2_{t-1|t-1}). $$ You would be able to use function composition to show if $P^2_{t|t} < P^1_{t|t}$. You would probably need to assume further that both models share the same initial-time state "prediction" distribution (or prior if you'd like). If $f_{2,t}$ always "increases" your last matrix by less than $f_{1,t}$, then using inductive reasoning will get you the desired result.

This seems to me intuitive for the reason that you mentioned: "...clear thatin[sic] (8) we keep on adding $\Sigma_{\eta}$ which we do not do under RLS," however I haven't shown it. We might be wrong.

Note 2:

You also mentioned that you're not sure which model is the true model. This should be a bigger concern than picking the model that yields the tighter confidence intervals. However, this isn't your question, so I won't say any more.

Note 3:

Another thing: all of this might be false without assuming they share the same observation covariance matrix. I suspect that your empirical observation won't always be true after running Kalman Filters (to obtain the filtering distributions) with estimated parameters plugged in. I haven't shown this either, though.