Does an accurate "noise" model help when using a Kalman filter?

Question

I've been trying to use a Kalman filter to estimate slope of a line (this is a simplified version of my problem for discussion). So basically time-varying regression.

State Equation: $$ \left[\begin{matrix} a_t \\ x_t \\ n_t \\ \end{matrix}\right] = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 0 & 0 & 0.9 \\ \end{bmatrix}\begin{bmatrix} a_{t-1} \\ x_{t-1} \\ n_{t-1} \\ \end{bmatrix} + \begin{bmatrix} \sigma_a \\ 0 \\ \sigma_n \\ \end{bmatrix}$$

Measurement Equation: $$ z_t = \begin{bmatrix} 0 & 1 & 1 \end{bmatrix}\begin{bmatrix} a_{t} \\ x_{t} \\ n_{t} \\ \end{bmatrix}$$

Now -- when I generate the data for simulation, I'm using three piece wise continuous straight lines for the signal with added AR1 noise of "driving" variance $\sigma_n^2$, as can be seen from the above equations. I use the same exact AR1 "noise" model in my Kalman filter equation for $n_t$. However, to compare, I also tried a model where the AR1 noise part was eliminated from the state equation and I just used an added term in the measurement equation for noise that had a variance of the simulated AR1 noise (so basically this just assumes a Gaussian white noise instead of AR1).

The results were virtually identical when observed the estimated slope...with the Gaussian noise assumption maybe even being a bit smoother (when I use the AR1 noise model in my state equations, there is a small high frequency component to the estimated slope). Note that in both cases, the driving noise for the slope $\sigma_a$ was optimized for min MSE of the line estimate. Plots for estimate with AR1 noise model used shown, with Gaussian noise assumption producing virtually identical plots and MSE:

So my question: does it really even matter if I accurately model the noise? I was expecting (or hoping) I'd get a better/smoother estimate of the slope when I modeled the noise exactly, but this is not the case. Is there something fundamental that someone can explain that I'm missing?

Answer 1

Here I will assume that your state space model is the general linear Gaussian one and that

$$y_{t} = Z_{t}\alpha_{t} + \epsilon_{t}, \;\;\;\;\;\; \epsilon_{t} \sim N(0, H_{t}),$$

and

$$\alpha_{t + 1} = T_{t}\alpha_{t} + R_{t}\eta_{t}, \;\;\;\;\;\ \eta_{t} \sim N(0, Q_{t}),\;\;\;\;\;\; \forall t = 1, \ldots, n.$$

where $\alpha_{t}$ is our unknown state vector at time step $t$, $y_{t}$ the observation vector etc. Our disturbances/error densities $\epsilon$ and $\eta$ are assumed to be Normally distributed and add noise to our state space model. For your case you do not model observation noise (no $H_{t}$).

If you look at the prediction step of the Kalman Filter recursion, you will see that the prediction step is linearly dependent on the covariance of the process noise $Q_{t}$. What does this mean? Well, crudely, this covariance will have an affect on how the state changes in each of the prediction steps (prediction and update) will change.

For the case above it is not the choice of noise-type that will be the driving factor, but it will be the magnitude of this noise. From the above it seems that you are assuming your measurement process is "perfect" in that you do not include an observation [noise] covariance term (this is fine). Due to the nature of the dependence on $Q_{t}$ if you take a too large value for this, the prediction step with change dramatically with each iteration; too small a value it won't update quickly enough for the optimal solution.

The answer here is to optimize for the covariances using a maximum likelihood method. This will iteratively filter the entire series (depending on your optimization routine) and provide the best values for $Q_{t}$ (and $H_{t}$ if required).

Answer 2

Got an answer from a professor at my university on this that I think makes sense:

"...it’s all about bandwidth overlap. The line ‘signal’ has VERY low (almost zero) BW. I would venture to say that, were your AR(1) noise to have BW parameter alpha ~ 0.98 – 0.99 you would find a more notable difference..."