I am reading the book "Time series analysis by state space methods" by J.Durbin and S.J. Koopman (second edition). On page 17, the book says, for local level models, the probability of one-step prediction error (of Kalman filter) $v_t$ at time $t$ is equal to the probability of observation $y_t$ at time $t$ conditioned on the previous observations. That is $p(v_t)=p(y_t|y_1,y_2,...,y_{t-1})$. This leads to that the prediction errors (or innovations) are mutually independent. But why the two probabilities are equal?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This is explained at the beginning of section 2.3.1 in the book. The idea is to transform the observations $y_1,\dots,y_n$ into the prediction errors $v_1,\dots,v_n$. According the recursions of the Kalman filter, this is a linear transformation:
\begin{eqnarray} \begin{array}{l} v_1 = y_1 - a_1 \\ v_2 = y_2 - a_2 - K_1(y_1-a_1) \\ v_2 = y_3 - a_1 - K_2(y_2-a_1) - K_1(1-K_2)(y_1-a_1) \\ \dots \end{array} \end{eqnarray}
where $a_1$ is the initial mean and $K_i$ are the Kalman gains.
Remember that if $f_Y(y)$ is the density of $y$, if the transformation $g(y)$ is applied then the density of $y$ can be written as the density of the transformed variable times the determinant of the Jacobian of the transformation, $f_Y(y) = f_X\left(g^{-1}(y)\right)\left|\frac{\partial g^{-1}(y)}{\partial y}\right|$. This result may be more familiar to you in the context of integration by means of a change of variable.
It can be checked that the determinant of the Jacobian of the transformation above is $1$. For example, taken $t=1,2,3$:
\begin{eqnarray} J=\left[ \begin{array}{ccc} \frac{\partial v_1}{\partial y_1} & \frac{\partial v_1}{\partial y_2} & \frac{\partial v_1}{\partial y_3} \\ \frac{\partial v_2}{\partial y_1} & \frac{\partial v_2}{\partial y_2} & \frac{\partial v_2}{\partial y_3} \\ \frac{\partial v_3}{\partial y_1} & \frac{\partial v_3}{\partial y_2} & \frac{\partial v_3}{\partial y_3} \end{array} \right] = \left[ \begin{array}{ccc} 1 & 0 & 0 \\ -K_1 & 1 & 0 \\ -K_1(1-K_2) & -K_2 & 1 \\ \end{array} \right] \,, \end{eqnarray} and the determinant is $|J|=1$. This leads to $f_Y(y) = f_X(x)$, i.e., $p(v_1)=p(y_1)$.
I think no much rigor is lost by putting it in a more intuitive way. If the observations $y$ are, say, Gaussian, then the distribution of the prediction errors $v$ will be Gaussian as well. After all, they are simply a linear transformation of $y$, where $a_1$ and $K_i$ do not depend on $y_1,\dots,y_n$. For example, taken $a_1$ as a fixed initial mean, it is easy to accept that $v_1 = y_1 - a_1$ is Gaussian when $y_1$ is Gaussian.
The above does not mean that the parameters of the distribution for $v$ are the same as those for $y$; i.e., the mean and covariance matrix of the prediction errors are not the same as the mean and covariance matrix of the observed data. The covariance matrix of the errors is precisely what this section of the book derives, which is useful to obtain an expression of the likelihood function that can be evaluated upon the output generated by the Kalman filter.
