I am working on an Bayesian problem from an OpenBugs example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC!. This is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain changepoint $x_k$.

The raw data looks like the following.

enter image description here

The basic setup is as following. (I am using $\sigma^2$ instead of precision $\tau$ as in the original example.) \begin{align*} Y_i \ & \sim \ N(\alpha + \beta_1 (x_i - x_k), \sigma^2), \; i = 1, \ldots, k \\ Y_i \ & \sim \ N(\alpha + \beta_2 (x_i - x_k), \sigma^2), \; i = k+1, \ldots, n \\ \end{align*}

Based on above plot, we might expect that $\beta_2$ is smaller than $\beta_1$ - deeper drop at later part, and the changepoint $x_k$ is around 0.

The example illustrates two parameterizations in terms of modeling the changepoint.

The first way is to assume the changepoint index $k$ a discrete uniform. The priors are \begin{align*} \alpha \ & \sim \ N(\mu_{\alpha}, \sigma^2_{\alpha}), \quad \sigma^2 \ \sim \ IG(a, b) \\ \beta_1, \beta_2 \ & \sim \ N(\mu_{\beta}, \sigma^2_{\beta}), \quad k \ \sim \ Unif\{1, n\} \end{align*} That is, the changepoint value $x_k$ is constrained to be one observed $x$ value.

The example points out one issue as following.

Note: alpha is E(Y) at the changepoint, so will be highly correlated with k. This may be a very poor parameterisation.

And as demonstrated in the example, "Results are hopeless - no mixing at all", and highly depend on the initial values of parameters (chain 1 (red): alpha = 0.2, k = 16; chain 2 (blue): alpha = 0.6, k = 8).

enter image description here

enter image description here

The second way is, instead of putting distribution on $k$, assuming the changepoint value $x_k$ a continuous uniform. Priors for other parameters are the same. \begin{align*} x_k \ \sim \ Unif(x_l, x_u) \end{align*} where $x_l, x_u$ can be predefined based on the range of observed $x$'s. In the example, $x_l=-1.3, x_u = 1.1$.

The results now did get better, as shown in following. enter image description here enter image description here

To fully understand this example, I have following questions.

  1. Why does the second parameterization work better? I assume that there is still "high correlation" issue (between $x_k$ and $\alpha$, as opposed to $k$ and $\alpha$ in first approach). The example shows the correlation between $x_k$ and $\alpha$ is -0.932941. So if the "high correlation" issue is not fixed, what does make the second way better?

  2. What is the main point of this example? What does the title mean by saying "illustration of how NOT to do MCMC"?

  3. In this particular example, are there even better parameterization approaches?

    Thank you very much for any comments.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.