# Finding conditional probability of an individual component of a joint distribution

Assume $$Y_1$$, $$Y_2$$, $$\ldots$$ ,$$Y_n$$ are random variables over a regular lattice indexed by $$i= 1,2,\ldots,n$$ where $$Y_i\in\{1,2,...,K\}$$. Let the probability of a particular configuration $$\textbf{y}= (y_1,y_2,...,y_n)$$ be given by

$$\mathsf P(\textbf{Y}=\textbf{y}) =C\cdot\text{exp}\left(\sum_{i=1}^n\alpha_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n\sum_{j\in N(i)}1(y_i=y_j)\right)$$

where $$C$$ is the normalizing constant, $$N(i)$$ is the set of neighbor points of $$i$$ and $$1(.)$$ is the indicator function. This model is known as Potts model and is popular in image analysis. Show that

$$\mathsf P\left(Y_i=k\mid Y_j=y_j, j\neq i\right)=\frac{\text{exp}\left(\alpha_{i,k}+\beta\sum_{j\in N(i)}1(y_j=k)\right)}{\sum_l\text{exp}\left(\alpha_{i,l}+\beta\sum_{j\in N(i)}1(y_j=l)\right)}$$

I first have some notational questions.

• Does the bolded $$\textbf{Y}$$ just mean that it's a vector?
• Does $$\mathsf P\left(Y_i=k\mid Y_j=y_j, j\neq i\right)$$ mean we know what a specific value for $$Y_j$$ is or does it mean we know what $$Y_j$$ is $$\textbf{for all}$$ $$j\neq i$$?

My try:

I discovered Brook's Lemma which says that if we let $$\textbf{y}_0=(y_{10},\ldots y_{n0})$$ be any fixed point in the support of $$p(.)$$ then

$$p(y_1,\ldots, y_n)=\frac{p(y_1\mid y_2,\ldots, y_n)}{p(y_{10}\mid y_2,\ldots y_n)}\cdots \frac{p(y_n\mid y_{10},\ldots, y_{n-1,0})}{p(y_{n0}\mid y_{10},\ldots, y_{n-1,0})}\cdot p(y_{10},\ldots, y_{n0})$$

which seems useful in that it relates the joint distribution in terms of its conditioned individual components. If my second notational question is correct, then I believe any one of the denominators in this expression gives the desired probability $$\mathsf P\left(Y_i=k\mid Y_j=y_j, j\neq i\right)$$ but it's not clear to me how to proceed or even if making use of this lemma is a viable approach. Any hints to get me going in the right direction would be greatly appreciated.

As a side question, how would one interpret the parameters $$\alpha_{i,y_i}$$ and $$\beta$$ in the joint distribution?

Note: I wasn't sure what to title this question so feel free to change it.

• About the questions: yes bold in $\mathbf Y$ is usually used to stress the rv is a vector. And the conditioning variable is made of all the components of $\mathbf Y$ except $Y_i$. Meaning that the realisations of all components $Y_j$ for $j\ne i$ are known. – Xi'an Oct 10 '19 at 20:52
• About the attempt: I call this representation the Hammersley-Clifford-Besag theorem but it is certainly not needed in this context:$$p(y_i|y_j,j\ne i)\propto p(y_1,\ldots,y_n)$$with a natural normalisation over all possible values of $y_i$. – Xi'an Oct 10 '19 at 20:55
• I have answered my own question but I'm not sure I did it right. – Remy Oct 11 '19 at 1:24

I think I may have figured it out. Bayes' Theorem gives that

$$\mathsf P(Y_i=k\mid Y_j=y_j, j\neq i) =\frac{\mathsf P(Y_i=k, Y_j=y_j, j\neq i)}{\mathsf P(Y_j=y_j, j\neq i)}$$

Integrating out $$Y_i$$ we get

$$\frac{C\cdot\text{exp}\left(\sum_{j\neq i}\alpha_{j,y_j}+\alpha_{i,k}+\frac{1}{2}\beta\sum_{j\neq i}\sum_{h\in N(j)}1(y_j=y_h)+\frac{1}{2}\beta\sum_{j\in N(i)}1(y_j=k)\right)}{\sum_{l} C\cdot\text{exp}\left(\sum_{j\neq i}\alpha_{j,y_j}+\alpha_{i,l}+\frac{1}{2}\beta\sum_{j\neq i}\sum_{h\in N(j)}1(y_j=y_h)+\frac{1}{2}\beta\sum_{j\in N(i)} 1(y_i=l)\right)}$$

which can be expanded as

$$\frac{\text{exp}\left(\sum_{j\neq i}\alpha_{j,y_j}+\alpha_{i,k}+\frac{1}{2}\beta\sum_{j\neq i}\sum_{h\in N(j)-i}1(y_j=y_h)+\frac{1}{2}\beta\sum_{h\in N(i)}1(y_h=k)+\frac{1}{2}\beta\sum_{j\in N(i)}1(y_j=k)\right)}{\sum_{l} \text{exp}\left(\sum_{j\neq i}\alpha_{j,y_j}+\alpha_{i,l}+\frac{1}{2}\beta\sum_{j\neq i}\sum_{h\in N(j)-i}1(y_j=y_h)+\frac{1}{2}\beta\sum_{h\in N(i)}1(y_h=l)+\frac{1}{2}\beta\sum_{j\in N(i)}1(y_j=l)\right)}$$

which simplifies to

$$\frac{\text{exp}\left(\alpha_{i,k}+\beta\sum_{j\in N(i)} 1(y_j=k)\right)}{\sum_l\text{exp}\left(\alpha_{i,l}+\beta\sum_{j\in N(i)} 1(y_j=l)\right)}$$

When $$\beta= 0$$, would it be fair to interpret the parameters $$\alpha_{i,k}$$ as multiplying the probability of observing the particular configuration by $$\text{exp}(\alpha_{i,k})$$?

For interpreting $$\beta$$, would it be fair to say that when the the number of neighbors each observation has in total increases by one, we would expect the probability of observing the particular configuration to multiply by $$\text{exp}(\beta)$$?