Gibbs Sampling Detecting Change point in time series

Question

I was reading through this one page paper on using Gibbs sampling for detecting a change point in a time series like data. While I understand the part where the $\lambda$ and $\phi$ are chosen from a gamma distribution I do not understand how the author generated the posterior distribution for $k$. My thinking is, if the first $k$ points came from a Poisson distribution then the probability of $P(k|\lambda,\phi,Y_i)$ $$ P(k|\lambda,\phi,Y_i) = \Pi_{i=1}^{k}\frac{e^{-\lambda}\lambda^{Y_i}}{Y_i!} \times \Pi_{i=k+1}^{N}\frac{e^{-\phi}\phi^{Y_i}}{Y_i!} $$ The author shows an expression of the form $\frac{A}{B}$ where $B$ is the normalizing term. This part I understand. However it is unclear to me how he found the numerator term which appears to be a log of $$ P(k|\lambda,\phi,Y_i) = \frac{\Pi_{i=1}^{k}\frac{e^{-\lambda}\lambda^{Y_i}}{Y_i!}}{\Pi_{i=k+1}^{N}\frac{e^{-\phi}\phi^{Y_i}}{Y_i!}} $$ How does he arrive at that? Any help appreciated. Thanks in advance

Answer 1

I believe the glitch is in your second equation, as the author's formula for $p(k|\textbf{y}, \lambda, \phi)$ doesn't depend on $N$. Manipulating his formula gives a quotient of products, but indexed slightly differently:

$$ e^{k(\phi - \lambda) + log(\frac{\lambda}{\phi})\sum_{i=1}^ky_i} = e^{k(\phi - \lambda)}e ^{ log(\frac{\lambda}{\phi})\sum_{i=1}^ky_i} $$

$$ = \frac{e^{-\lambda k}}{e^{-\phi k}}\left(\frac{\lambda}{\phi}\right)^{\sum_{i=1}^ky_i} $$

$$ = \frac{e^{-\lambda k}}{e^{-\phi k}}\left(\frac{\lambda^{\sum_{i=1}^ky_i}}{\phi^{\sum_{i=1}^ky_i}}\right) $$

$$ = \frac{\Pi_{i=1}^{k}\frac{e^{-\lambda}\lambda^{y_i}}{y_i!}}{\Pi_{i=1}^{k}\frac{e^{-\phi}\phi^{y_i}}{y_i!}} $$

To find our way to this equation, first factor the joint probability $p(\lambda, \phi, k, \textbf{y})$ two ways:

$$ p(\lambda, \phi, k, \textbf{y}) = p(\textbf{y}|\lambda, \phi, k)p(\lambda, \phi, k) = p(k|\lambda, \phi, \textbf{y})p(\lambda, \phi, \textbf{y}) $$

To solve for $p(k|\lambda, \phi, \textbf{y})$, divide both factored probabilities by $p(\lambda, \phi, \textbf{y})$:

$$ p(k|\lambda, \phi, \textbf{y}) = \frac{p(\textbf{y}|\lambda, \phi, k)p(\lambda, \phi, k)}{p(\lambda, \phi, \textbf{y})} $$

As succinctly put on p. 4 of this similar treatment of the same data and approach, "Trick: in conditional posterior distribution, the parameters that are conditioned on are treated as constants." Using this and the fact that $\lambda, \phi$, and $k$ have independent priors, and the prior for $k$ is a uniform constant, we now have a slight variation on your first equation:

$$ p(k|\lambda, \phi, \textbf{y}) \propto p(\textbf{y}|\lambda, \phi, k) $$ $$ p(k|\lambda, \phi, \textbf{y}) \propto \Pi_{i=1}^{k}\frac{e^{-\lambda}\lambda^{y_i}}{y_i!} \times \Pi_{i=k+1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!} $$ Because $\phi$ and $\textbf{y}$ are known, we can divide both sides by $\prod_{i=1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!}$ and maintain proportionality.

$$ p(k|\lambda, \phi, \textbf{y}) \propto \frac{\prod_{i=1}^{k}\frac{e^{-\lambda}\lambda^{y_i}}{y_i!} \times \prod_{i=k+1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!}}{\prod_{i=1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!}} = \frac{\prod_{i=1}^{k}\frac{e^{-\lambda}\lambda^{y_i}}{y_i!} \times \prod_{i=k+1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!}}{\prod_{i=1}^{k}\frac{e^{-\phi}\phi^{y_i}}{y_i!} \times \prod_{i=k+1}^{N}\frac{e^{-\phi}\phi^{y_i}}{y_i!}} $$

The rightmost products in the numerator and the denominator cancel, and we've solved for for $p(k|\lambda,\phi, \textbf{y})$ up to a proportionality constant:

$$ p(k|\lambda, \phi, \textbf{y}) \propto \frac{\Pi_{i=1}^{k}\frac{e^{-\lambda}\lambda^{y_i}}{y_i!}}{\Pi_{i=1}^{k}\frac{e^{-\phi}\phi^{y_i}}{y_i!}} = e^{k(\phi - \lambda) + log(\frac{\lambda}{\phi})\sum_{i=1}^ky_i} $$

To yield a proper conditional posterior, we normalize over $k$, giving the author's formula.