3
$\begingroup$

Question: Consider the local linear trend model given by: \begin{align*} y_t = \mu_t + \tau \varepsilon_t \ \cdots \ \text{Observation equation} \\ \mu_{t+1} = \phi \mu_t + \eta_t \ \cdots \ \text{State equation} \end{align*} for $t = 1, 2, \cdots, T$, where $(\varepsilon_t, \eta_t)'$ is independent of $\mu_k$ for $k \le t$ and where: \begin{align*} \begin{bmatrix} \varepsilon_t \\ \eta_t \end{bmatrix} \stackrel{i.i.d}{\sim} N\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\right) \end{align*} and \begin{align*} \mu_1 \sim N\left(0, \frac{1}{1-\phi^2} \right) \end{align*}

(Note everything is a scalar quantity in this question).

Consider a Bayesian analysis of this model under the prior distribution given by: \begin{align*} p(\theta) \propto \frac{1}{\tau} \ \text{for} \ -\infty < \phi < \infty \ \text{and} \ 0<\tau<\infty \end{align*} where $\theta = (\phi, \tau)$.

Devise a Gibbs sampler to sample from the joint posterior distribution, $p(\mu_{1:T}, \theta \mid y_{1:T})$ where the notation $\mu_{1:T}$ denotes $(\mu_1, \mu_2, \cdots, \mu_T)$ and similarly, $y_{1:T}$ denotes $(y_1, y_2, \cdots, y_T)$.


My Working So Far:

The joint posterior distribution is given by: \begin{align*} p(\mu_{1:T}, \theta \mid y_{1:T}) & \propto p(\mu_{1:T}, \theta, y_{1:T}) \\ & = \underbrace{p(y_{1:T} \mid \mu_{1:T}, \theta)}_{\text{'likelihood'}}\underbrace{p(\mu_{1:T} \mid \theta)p(\theta)}_{\text{prior}} \\ & = \left[\prod_{t=1}^{T} p(y_t \mid \mu_t, \theta)\right]\left[\prod_{t=1}^{T-1}p(\mu_{t+1} \mid \mu_t, \theta) \right]p(\mu_1 \mid \theta)p(\theta) \ \ \cdots \ \ (1) \end{align*} Since $y_t \mid \mu_t, \theta \sim N(\mu_t, \tau^2)$ for $t=1, 2, \cdots, T$, the pdf is given by: \begin{gather*} p(y_t \mid \mu_t, \theta) = \left(2\pi \tau^2\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2\tau^2}\left(y_t - \mu_t\right)^2 \right] \end{gather*} Similarly, $\mu_{t+1} \mid \mu_t, \theta \sim N\left(\phi \mu_t, 1 \right)$, so the pdf is given by: \begin{gather*} p(\mu_{t+1} \mid \mu_t, \theta) = \left(2\pi\right)^{-\frac{1}{2}} \exp\left[-\frac{1}{2}\left(\mu_{t+1} - \phi \mu_t \right)^2 \right] \end{gather*} We know that $\mu_1 \mid \theta \sim N\left(0, \frac{1}{1-\phi^2} \right)$, so the pdf is given by: \begin{gather*} p(\mu_1 \mid \theta) = \left(2\pi\left(\frac{1}{1-\phi^2} \right) \right)^{-\frac{1}{2}} \exp\left[-\frac{\mu_1^2}{2\left(\frac{1}{1-\phi^2}\right)} \right] \end{gather*} Finally, we are given that $p\left(\theta\right) \propto \frac{1}{\tau}$.

Substituting all of the above into Eqn. $(1)$, yields the joint posterior distribution: \begin{align*} p(\mu_{1:T}, \theta \mid y_{1:T}) & \propto \left[\prod_{t=1}^{T} \left(2\pi \tau^2\right)^{-\frac{1}{2}}\exp\left[-\frac{1}{2\tau^2}\left(y_t - \mu_t\right)^2 \right]\right]\left[\prod_{t=1}^{T-1}\left(2\pi\right)^{-\frac{1}{2}} \exp\left[-\frac{1}{2}\left(\mu_{t+1} - \phi \mu_t \right)^2 \right] \right] \\ & \times \left(2\pi\left(\frac{1}{1-\phi^2} \right) \right)^{-\frac{1}{2}} \exp\left[-\frac{\mu_1^2}{2\left(\frac{1}{1-\phi^2}\right)} \right] \left( \frac{1}{\tau}\right) \end{align*}

I will implement a "blocked" Gibbs sampler sampling $\mu_{1:T}^{(i)}$ together, as follows:

For $i = 1, 2, \cdots, M$, sample: \begin{align*} \mu_{1:T}^{(i)} & \sim \mu_{1:T} \mid \phi^{(i-1)}, \tau^{(i-1)}, y_{1:T} \ \ \cdots \ \ (2)\\ \phi^{(i)} & \sim \phi \mid \mu_{1:T}^{(i)}, \tau^{(i-1)}, y_{1:T} \ \ \cdots \ \ (3) \\ \tau^{(i)} & \sim \tau \mid \mu_{1:T}^{(i)}, \phi^{(i)}, y_{1:T} \ \ \cdots \ \ (4) \end{align*}

Sampling from $(2)$ is straightforward by using the Forward Filter Backwards Sampling (FFBS)

My Query: I am stuck on how to sample from $(3)$ and $(4)$, in order to use a Gibbs sampler on $(3)$ and $(4)$, we need to find the full conditional of $\phi \mid \mu_{1:T}, \tau, y_{1:T}$ and $\tau \mid \mu_{1:T}, \phi, y_{1:T}$, but how do you find these full conditionals? I do not see any obvious way by examining the joint posterior distribution. Perhaps a Metropolis Hastings subchain could work? But then what should I pick for my candidate density for the $\phi$ and $\tau$?

Thanks in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.