I am working on an Bayesian problem from an OpenBugs example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC!. This is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain changepoint $x_k$.
The raw data looks like the following.
The basic setup is as following. (I am using $\sigma^2$ instead of precision $\tau$ as in the original example.) \begin{align*} Y_i \ & \sim \ N(\alpha + \beta_1 (x_i - x_k), \sigma^2), \; i = 1, \ldots, k \\ Y_i \ & \sim \ N(\alpha + \beta_2 (x_i - x_k), \sigma^2), \; i = k+1, \ldots, n \\ \end{align*}
Based on above plot, we might expect that $\beta_2$ is smaller than $\beta_1$ - deeper drop at later part, and the changepoint $x_k$ is around 0.
The example illustrates two parameterizations in terms of modeling the changepoint.
The first way is to assume the changepoint index $k$ a discrete uniform. The priors are \begin{align*} \alpha \ & \sim \ N(\mu_{\alpha}, \sigma^2_{\alpha}), \quad \sigma^2 \ \sim \ IG(a, b) \\ \beta_1, \beta_2 \ & \sim \ N(\mu_{\beta}, \sigma^2_{\beta}), \quad k \ \sim \ Unif\{1, n\} \end{align*} That is, the changepoint value $x_k$ is constrained to be one observed $x$ value.
The example points out one issue as following.
Note: alpha is E(Y) at the changepoint, so will be highly correlated with k. This may be a very poor parameterisation.
And as demonstrated in the example, "Results are hopeless - no mixing at all", and highly depend on the initial values of parameters (chain 1 (red): alpha = 0.2, k = 16; chain 2 (blue): alpha = 0.6, k = 8).
The second way is, instead of putting distribution on $k$, assuming the changepoint value $x_k$ a continuous uniform. Priors for other parameters are the same. \begin{align*} x_k \ \sim \ Unif(x_l, x_u) \end{align*} where $x_l, x_u$ can be predefined based on the range of observed $x$'s. In the example, $x_l=-1.3, x_u = 1.1$.
The results now did get better, as shown in following.
To fully understand this example, I have following questions.
Why does the second parameterization work better? I assume that there is still "high correlation" issue (between $x_k$ and $\alpha$, as opposed to $k$ and $\alpha$ in first approach). The example shows the correlation between $x_k$ and $\alpha$ is -0.932941. So if the "high correlation" issue is not fixed, what does make the second way better?
What is the main point of this example? What does the title mean by saying "illustration of how NOT to do MCMC"?
In this particular example, are there even better parameterization approaches?
Thank you very much for any comments.
