# Distribution of conditional posterior for Gibbs sampling

The following is a description of how the authors (Yongning Wang & Ruey S. Tsay) of this (2019) paper Clustering Multiple Time Series with Structural Breaks want to perform Gibbs sampling to obtain cluster memberships. i.e. let's say c can become 1,2 or 3, meaning an individual n can belong to cluster 1,2 or 3. If I perform Gibbs sampling I get stuck, as the posterior distribution they describe is non-standard. How do they get to the probability statement? It seems like the big term before $$\eta_{sc}$$ is simply a scalar and does not influence individual draws.

• The $n^{th}$ individual is sampled to group $c$ with a probability $p$; $p$ is not the group, but the probability of choosing that group. Jun 13, 2022 at 14:42
• Yes, maybe I was not clear. I meant to ask what then the distribution is that I am to sample group c from given these probabilities. Edit : thanks for your response! Jun 13, 2022 at 14:44
• @user773674 the categorical distribution with the given probability. To be more precise, calculate the expressions for c = 1,2,3 and store those in a vector. Then, divide that vector by its sum. Then, sample (1,2,3) with probability given by that vector. Jun 13, 2022 at 14:46
• @LucasRoberts (the article is available on Scihub ;)). I just went through the paper to double check, looks like this article is describing a fully Bayesian procedure, with the exception of the total number of clusters K, which is selected via marginal likelihood maximization (aka empirical bayes). Jun 13, 2022 at 15:27
• It seems to me that the probability only depends on $$\eta_{sk}$$. Note that k and c are the same here, the appendix is weird with notation. As all variables within the product over all k's for all individuals n=1,...,N, doesn't influence the proportions between the probability of i.e. let k in {1,2,3}. Then $$p(c_{n,(s)}=1)$$, $$p(c_{n,(s)}=2)$$, $$p(c_{n,(s)}=3)$$ only depend on the eta term, as the terms with sk subscript in the product remain the same no? EDIT: all conditioning left out of notation obviously Jun 13, 2022 at 20:23

The time-series model in the paper is

1. multivariate with $$N$$ time series of duration $$T$$
2. regime-switching, meaning that the model parameters are changing at random times (with indicators $$s_t$$), with an unknown number $$S$$ of switches
3. clustered, meaning that during one regime (time interval) $$s$$, the components of the time series are driven by one of $$K$$ models, with all components in the same cluster $$k$$ following the same model (sharing the same parameter $$\theta_{sk}$$

When considering a Bayesian modelling of this setting, simulating the parameters and the latent variables (regimes and clusters) can be done by Gibbs sampling. For the cluster allocation, this can be done conditional on the regimes $$\mathbf s^T$$ and the parameters, since the clustering is independent between regimes. Each component $$n$$ of the time series vector can then be allocated to one of the $$K$$ clusters, conditional on the other components, with a probability $$\mathbb P(c_{n,s}=\kappa)\propto \eta_{s\kappa} \prod_k \int f(\mathbf y_{ns}|\theta_{s\kappa})\pi(\theta s_k|\mathbf y_{-n,s})\,\text d\theta_{sk}\tag{1}$$ that has integrated out the first level parameters, thanks to the conjugacy of the priors (cf. (A-19) in the Appendix A-3). Since the rhs is available in closed form, the simulation of $$c_{n,s}$$ is straightforward as a discrete distribution on $$\{1,2,\ldots,K\}$$, as indicated in John Madden's comment.

calculate the expressions for c = 1,2,3 and store those in a vector. Then, divide that vector by its sum. Then, sample (1,2,3) with probability given by that vector.

This is one step of a partly integrated Gibbs sampler that is perfectly valid. Note that, due to this integration of the first level group- and cluster-dependent parameters, the components $$\mathbf y_{ms}$$ are no longer (conditionally) independent, which is why they all appear in the right hand side of (1).

• Would you agree the probabilities given are problematic since we do not obtain a probability for any group c that is not in $C_{s}$ i.e. is not present during regime s. And for the c's that are present in $C_{s}$ probability is only scaled by the big term and depends on $\eta_{sk}$. Which is modelled independently? Jun 19, 2022 at 17:48