2
$\begingroup$

I am considering particle filtering methods for the estimation of static and dynamic parameters. For the static parameters $\theta$, Liu and West (page 7, equation 3.1) describe an "artificial" perturbation: $$ \theta_{t+1} = \theta_t + \zeta_{t+1} $$ $$ \zeta_{t+1} \sim N(0, W_{t+1}) $$ in order to generate a sample from $p(\theta_{t+1} | D_t)$ before updating the sample based on the likelihood of the new data at time $t+1$ to obtain a sample from $p(\theta_{t+1} | D_{t+1})$. Note that the notation $\theta_t$ is used to mean that it denote the time $t$ posterior of $\theta$ and not that $\theta$ is a time-varying parameter.

Now adding this artificial perturbation increases the variance of the sample and so the estimate of the true posterior is over-dispersed. They describe a method in which you may perturb the samples without increasing the sample variance. That is, so: $$ V(\theta_{t+1} | D_t) = V(\theta_t | D_t) + W_{t+1} + 2C(\theta_t, \zeta_{t+1} | D_t) = V(\theta_t | D_t) $$ which implies: $$ C(\theta_t, \zeta_{t+1} | D) = -W_{t+1}/2 $$ (this is all from page 9 of Liu and West). They then state that approximating joint normality of $(\theta_t, \zeta_{t+1} | D_t)$ then implies: $$ \theta_{t+1} | \theta_t \sim N(A_{t+1} \theta_t + (1-A_{t+1}) \bar{\theta_t}, (1-A_{t+1}^2)V_t) $$ where: $$ A_{t+1} = I - W_{t+1}V_t^{-1}/2 $$ and the idea is that if $p(\theta_t | D_t)$ has finite mean $\bar{\theta}_t$ and variance matrix $V_t$, then the mean and variance matrix of $p(\theta_{t+1} | D_t)$ is also $\bar{\theta}_t$ and $V_t$.

My question: How do you derive this form of $\theta_{t+1} | \theta_t$? It makes sense but is not obvious (to me) how to get there, any advice whatsoever would be appreciated.

$\endgroup$

2 Answers 2

3
$\begingroup$

Answering my own question based on the help from @Taylor. A very nice proof of the results used is here, from which I have taken some notation.

For a multivariate normal vector $\boldsymbol{Y} \sim N(\boldsymbol{\mu}, \boldsymbol{\Sigma})$, consider partitioning $\boldsymbol{\mu}$ and $\boldsymbol{Y}$ into: \begin{equation} \begin{aligned} \boldsymbol\mu = \begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2 \end{bmatrix} \end{aligned} \qquad \text{and} \qquad \begin{aligned} \boldsymbol{Y}= \begin{bmatrix} \boldsymbol{y}_1 \\ \boldsymbol{y}_2 \end{bmatrix} \end{aligned} \end{equation} with a similar partition of $\boldsymbol{\Sigma}$ into: $$ \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix}. $$ The conditional distribution of the first partition given the second is then: $$ \boldsymbol{y}_1 | \boldsymbol{y}_2 \sim N(\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2), \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12}{\boldsymbol{\Sigma}_{22}}^{-1} \boldsymbol{\Sigma}_{21}). $$

In this problem: $$ \begin{aligned} C(\boldsymbol{\theta}_{t+1}, \boldsymbol{\theta}_t | D_t) &= E(\boldsymbol{\theta}_{t+1} \boldsymbol{\theta}_t^\intercal | D_t) - E(\boldsymbol{\theta}_{t+1} | D_t)E(\boldsymbol{\theta}_t | D_t)^\intercal \\ &= E((\boldsymbol{\theta}_t + \boldsymbol{\zeta}_{t+1}) \boldsymbol{\theta}_t^\intercal | D_t) - \bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal \\ &= E(\bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal + \boldsymbol{\zeta}_{t+1} \boldsymbol{\theta}_t^\intercal | D_t) - \bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal \\ &= E(\bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal | D_t) + E(\boldsymbol{\zeta}_{t+1} \boldsymbol{\theta}_t^\intercal | D_t) - \bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal \\ &= (\mathbf{V}_t + \bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal ) + (-\frac{1}{2} \mathbf{W}_{t+1} + 0) - \bar{\boldsymbol{\theta}_t}\bar{\boldsymbol{\theta}_t}^\intercal \\ &= \mathbf{V}_t -\frac{1}{2} \mathbf{W}_{t+1} \\ \boldsymbol{\mu}_1 &= \boldsymbol{\mu}_2 = \bar{\boldsymbol{\theta}}_t \\ \boldsymbol{\Sigma}_{11} &= \boldsymbol{\Sigma}_{22} = \mathbf{V}_t \\ \boldsymbol{\Sigma}_{12} &= \boldsymbol{\Sigma}_{21} = \mathbf{V}_t -\frac{1}{2} \mathbf{W}_{t+1} \end{aligned} $$ and so the conditional normal evolution is: $$ \boldsymbol{\theta}_{t+1} | \boldsymbol{\theta}_t \sim N(\bar{\boldsymbol{\theta}}_t + (\mathbf{V}_t -\frac{1}{2} \mathbf{W}_{t+1}) \mathbf{V}_t^{-1} (\boldsymbol{\theta}_t -\bar{\boldsymbol{\theta}}_t), \mathbf{V}_t - (\mathbf{V}_t -\frac{1}{2} \mathbf{W}_{t+1}) \mathbf{V}_t^{-1} (\mathbf{V}_t -\frac{1}{2} \mathbf{W}_{t+1})) $$ which simplifies to: $$ \boldsymbol{\theta}_{t+1} | \boldsymbol{\theta}_t \sim N(\mathbf{A}_t \boldsymbol{\theta}_t + (\mathbf{I} - \mathbf{A}_t) \bar{\boldsymbol{\theta}}_t, (\mathbf{I} - \mathbf{A}_t^2) \mathbf{V}_t) $$ where: $$ \mathbf{A}_t = \mathbf{I} - \frac{1}{2} \mathbf{W}_{t+1} \mathbf{V}_t^{-1}. $$

$\endgroup$
1
+250
$\begingroup$

This follows from a property of multivariate normal random vectors. Here's a link: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Conditional_distributions

$\endgroup$
1
  • 1
    $\begingroup$ Didn't know/forgot about that result. I've worked it all out now - thanks! $\endgroup$
    – Jeff
    Jul 27, 2015 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.