3
$\begingroup$

An intuitive explanation with some math derivation is preferred. Pure mathematical equations are not helpful.

Update: By "Q" I mean the process error, which is the variance of wj in the following equations. By "R" I mean the measurement error, which is the variance of vj in the following equations. More details can be found in the link in the bottom.

enter image description here

Q and R will be used in the recursive calculation as follows.

enter image description here

Reference: http://www.swarthmore.edu/NatSci/echeeve1/Ref/Kalman/ScalarKalman.html

Please note I am only asking about one-dimensional Kalman Filter, ie. only estimating one variable.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Your question is unclear. Please define what Q and R are (I've seen dozens of different formulations of the Kalman Filter, with different notation, and often organized differently) and describe which version of the KF you're using. $\endgroup$
    – Glen_b
    Jan 4, 2015 at 18:14
  • $\begingroup$ Are you assuming that A, B and H matrices are known? $\endgroup$
    – Aksakal
    Jan 8, 2015 at 16:37
  • $\begingroup$ Yes, these are assumed to be known. $\endgroup$
    – nouveau
    Jan 8, 2015 at 20:39
  • $\begingroup$ Are you asking about what are Q and R? Or How are they specified? $\endgroup$
    – CroCo
    Jan 22, 2015 at 2:25

1 Answer 1

Reset to default
4
$\begingroup$

I will be relatively crude with my explanation in an attempt to forego some mathematical nuances. Here I will assume that your state space model is the general linear Gaussian one and that

$$y_{t} = Z_{t}\alpha_{t} + \epsilon_{t}, \;\;\;\;\;\; \epsilon_{t} \sim N(0, H_{t}),$$

and

$$\alpha_{t + 1} = T_{t}\alpha_{t} + R_{t}\eta_{t}, \;\;\;\;\;\ \eta_{t} \sim N(0, Q_{t}),\;\;\;\;\;\; \forall t = 1, \ldots, n.$$

where $\alpha_{t}$ is our unknown state vector at time step $t$, $y_{t}$ the observation vector etc. Our disturbances/error densities $\epsilon$ and $\eta$ are assumed to be Normally distributed and add noise to our state space model.

If you look at the prediction step of the Kalman Filter recursion, you will see that the prediction step is linearly dependent on the covariance of the process noise $Q_{t}$, and that the update step is linearly dependent on the inverse of $H_{t}$. What does this mean? Well, crudely, the covariances will determine how the state changes in each of these steps (prediction and update) will change.

Let us take a 1D univariate model and neglect $H_{t}$ for now. If you take a too large value for $Q_{t}$, the prediction step with change dramatically with each iteration. If the value of $H_{t}$ is set too large, the model will be slow to respond to new observations. So it is clear that the choice of the covariances will have a direct effect on the predictions/updates and model performance. So, what we are after are optimal values for $H_{t}$ and $Q_{t}$, that is values that will optimise how the state changes during the prediction/update recursion in order to represent the true underlying process.

The optimisation of these parameters is what the "parameter estimation" process does. It attempts to maximise the likelihood function (LF) (often simply the likelihood) which is a function of the parameters of a statistical model. The likelihood of a set of parameter values and given outcomes, is equal to the probability of those observed outcomes given those parameter values. Again crudely, during the parameter estimation process the Kalman Filter is run many times whilst updating $H_{t}$, $Q_{t}$ and attempting to maximise the LF. When the LF is maximised (converged to a constant value) we have the optimum values of our covariances/parameters for that state space model.

I hope this helps.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks Killercam. I might need two more days to understand your post. This week is hectic for me. $\endgroup$
    – nouveau
    Jan 8, 2015 at 20:41
  • 2
    $\begingroup$ (+1) Just one detail, the purpose in parameter estimation is to maximise the likelihood function in order to get those parameter values that make the observed data more likely. Since by default most optimization algorithms minimise the objective function, in practice it is convenient to minimise the negative of the likelihood function, which is what I think you have in mind. $\endgroup$
    – javlacalle
    Jan 8, 2015 at 21:03
  • $\begingroup$ Yes, you are right, I have got my min and max mixed up here.. I will edit tomorrow. $\endgroup$
    – MoonKnight
    Jan 8, 2015 at 22:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.