Question: Consider the local linear trend model given by: \begin{align*} y_t = \mu_t + \tau \varepsilon_t \ \cdots \ \text{Observation equation} \\ \mu_{t+1} = \phi \mu_t + \eta_t \ \cdots \ \text{State equation} \end{align*} for $t = 1, 2, \cdots, T$, where $(\varepsilon_t, \eta_t)'$ is independent of $\mu_k$ for $k \le t$ and where: \begin{align*} \begin{bmatrix} \varepsilon_t \\ \eta_t \end{bmatrix} \stackrel{i.i.d}{\sim} N\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\right) \end{align*} and \begin{align*} \mu_1 \sim N\left(0, \frac{1}{1-\phi^2} \right) \end{align*}

(Note everything is a scalar quantity in this question).

Consider a Bayesian analysis of this model under the prior distribution given by: \begin{align*} p(\theta) \propto \frac{1}{\tau} \ \text{for} \ -\infty < \phi < \infty \ \text{and} \ 0<\tau<\infty \end{align*} where $\theta = (\phi, \tau)$.

Devise a Gibbs sampler to sample from the joint posterior distribution, $p(\mu_{1:T}, \theta \mid y_{1:T})$ where the notation $\mu_{1:T}$ denotes $(\mu_1, \mu_2, \cdots, \mu_T)$ and similarly, $y_{1:T}$ denotes $(y_1, y_2, \cdots, y_T)$.

My Working So Far:

The joint posterior distribution is given by: \begin{align*} p(\mu_{1:T}, \theta \mid y_{1:T}) & \propto p(\mu_{1:T}, \theta, y_{1:T}) \\ & = \underbrace{p(y_{1:T} \mid \mu_{1:T}, \theta)}_{\text{'likelihood'}}\underbrace{p(\mu_{1:T} \mid \theta)p(\theta)}_{\text{prior}} \\ & = \left[\prod_{t=1}^{T} p(y_t \mid \mu_t, \theta)\right]\left[\prod_{t=1}^{T-1}p(\mu_{t+1} \mid \mu_t, \theta) \right]p(\mu_1 \mid \theta)p(\theta) \ \ \cdots \ \ (1) \end{align*} Since $y_t \mid \mu_t, \theta \sim N(\mu_t, \tau^2)$ for $t=1, 2, \cdots, T$, the pdf is given by: \begin{gather*} p(y_t \mid \mu_t, \theta) = \left(2\pi \tau^2\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2\tau^2}\left(y_t - \mu_t\right)^2 \right] \end{gather*} Similarly, $\mu_{t+1} \mid \mu_t, \theta \sim N\left(\phi \mu_t, 1 \right)$, so the pdf is given by: \begin{gather*} p(\mu_{t+1} \mid \mu_t, \theta) = \left(2\pi\right)^{-\frac{1}{2}} \exp\left[-\frac{1}{2}\left(\mu_{t+1} - \phi \mu_t \right)^2 \right] \end{gather*} We know that $\mu_1 \mid \theta \sim N\left(0, \frac{1}{1-\phi^2} \right)$, so the pdf is given by: \begin{gather*} p(\mu_1 \mid \theta) = \left(2\pi\left(\frac{1}{1-\phi^2} \right) \right)^{-\frac{1}{2}} \exp\left[-\frac{\mu_1^2}{2\left(\frac{1}{1-\phi^2}\right)} \right] \end{gather*} Finally, we are given that $p\left(\theta\right) \propto \frac{1}{\tau}$.

Substituting all of the above into Eqn. $(1)$, yields the joint posterior distribution: \begin{align*} p(\mu_{1:T}, \theta \mid y_{1:T}) & \propto \left[\prod_{t=1}^{T} \left(2\pi \tau^2\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2\tau^2}\left(y_t - \mu_t\right)^2 \right]\right]\left[\prod_{t=1}^{T-1}\left(2\pi\right)^{-\frac{1}{2}} \exp\left[-\frac{1}{2}\left(\mu_{t+1} - \phi \mu_t \right)^2 \right] \right] \\ & \times \left(2\pi\left(\frac{1}{1-\phi^2} \right) \right)^{-\frac{1}{2}} \exp\left[-\frac{\mu_1^2}{2\left(\frac{1}{1-\phi^2}\right)} \right] \left( \frac{1}{\tau}\right) \end{align*}

I will implement a "blocked" Gibbs sampler sampling $\mu_{1:T}^{(i)}$ together, as follows:

For $i = 1, 2, \cdots, M$, sample: \begin{align*} \mu_{1:T}^{(i)} & \sim \mu_{1:T} \mid \phi^{(i-1)}, \tau^{(i-1)}, y_{1:T} \ \ \cdots \ \ (2)\\ \phi^{(i)} & \sim \phi \mid \mu_{1:T}^{(i)}, \tau^{(i-1)}, y_{1:T} \ \ \cdots \ \ (3) \\ \tau^{(i)} & \sim \tau \mid \mu_{1:T}^{(i)}, \phi^{(i)}, y_{1:T} \ \ \cdots \ \ (4) \end{align*}

Sampling from $(2)$ is straightforward by using the Forward Filter Backwards Sampling (FFBS)

My Query: I am stuck on how to sample from $(3)$ and $(4)$, in order to use a Gibbs sampler on $(3)$ and $(4)$, we need to find the full conditional of $\phi \mid \mu_{1:T}, \tau, y_{1:T}$ and $\tau \mid \mu_{1:T}, \phi, y_{1:T}$, but how do you find these full conditionals? I do not see any obvious way by examining the joint posterior distribution. Perhaps a Metropolis Hastings subchain could work? But then what should I pick for my candidate density for the $\phi$ and $\tau$?

Thanks in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.