As an extension of my previous question which is described below...

Assume $Y_1$, $Y_2$, $\ldots$ ,$Y_n$ are random variables over a regular lattice indexed by $i= 1,2,\ldots,n$ where $Y_i\in\{1,2,...,K\}$. Let the probability of a particular configuration $\textbf{y}= (y_1,y_2,...,y_n)$ be given by

$$\mathsf P(\textbf{Y}=\textbf{y}) =C\cdot\text{exp}\left(\sum_{i=1}^n\alpha_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n\sum_{j\in N(i)}1(y_i=y_j)\right)$$

where $C$ is the normalizing constant, $N(i)$ is the set of neighbor points of $i$ and $1(.)$ is the indicator function. This model is known as Potts model and is popular in image analysis.

Let us instead of observing $\textbf{Y}$,

we observe $Z_1,Z_2,\ldots,Z_n$ which are conditionally (on $\textbf{Y}$) independent with $$Z_i\mid\textbf{Y}\stackrel{d}{=}Z_i\mid Y_i\sim f_{Y_i}(.)$$ where $f_k(.)$ are known distributions for $k= 1,2,\ldots,K$. Show that

$$\mathsf P(\textbf{Y}\mid\textbf{Z}) =\tilde{C}\cdot\text{exp}\left(\sum_{i=1}^n\tilde{\alpha}_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n\sum_{j\in N(i)}1(y_i=y_j)\right)$$

for appropriate choices of $\tilde{\alpha}_{i,k}$ and a new normalizing constant $\tilde{C}$.

My thoughts:

It's not clear to me what it means, in words, for $Z_1,Z_2,\ldots,Z_n$ to be conditionally independent (on $\textbf{Y}$). Does this mean $Y_i$ will give us just as much information about $Z_i$ as $\textbf{Y}$?

I thought it may be useful to use Bayes' Theorem. We have

$$\begin{align*} \mathsf P(\textbf{Y}\mid\textbf{Z}) &=\frac{\mathsf P(\textbf{Z}\mid\textbf{Y})\cdot\mathsf P(\textbf{Y})}{P(\textbf{Z})}\\\\ &\propto\mathsf P(\textbf{Z}\mid\textbf{Y})\cdot\mathsf P(\textbf{Y})\\\\ &=\left(\prod_{i=1}^n \mathsf P(Z_i\mid Y_i)\right)\mathsf P(\textbf{Y}) \end{align*}$$

However, I'm not sure if this is correct or if this is even the right approach. Any suggestions on how I can proceed or alternative approaches would be greatly appreciated.


2 Answers 2


It's not clear to me what it means, in words, for Z1,Z2,…,Zn to be conditionally independent (on Y).

It means that given $Y=y$ all your $Z_i's $ are statistically independent of each other. Otherwise they could be correlated. An example could be :

Suppose there is an algorithm which classifies if an image has a car in it or not. Every time it runs on a bunch of images you generate a score for it represented by the variable $Z_i$. Let $Y$ be 1 if the algorithm learns over time after knowing the correct results (a reinforcement machine learning kind of an algorithm) and let $Y$ be 0 otherwise.

Now given $Y=0$ you really won't expect your algorithm to improve in performance over time, or in other words $Z_1$ wouldn't have much to say about $Z_2$. However, for a reinforcement type of an algorithm (when $Y=1 $), you would expect the scores to improve over time in general $Z_1 < Z_{1000}$ is more likely compared to $Z_{1000} < Z_1$.

Hence conditional on $Y=0$, $ \quad Z_i's$ are independent.


We have that

$$\begin{align*} \mathsf{P}(\textbf{Y}\mid\textbf{Z}) &=\frac{\mathsf P(\textbf{Z}\mid\textbf{Y})\cdot\mathsf P(\textbf{Y})}{\mathsf P(\textbf{Z})}\\\\ &=C_0\cdot \mathsf P(\textbf{Z}\mid\textbf{Y})\cdot\mathsf P(\textbf{Y})\\\\ &=C_0\prod_{i=1}^n f_{Y_i}(\cdot)\cdot C\cdot\text{exp}\left(\sum_{i=1}^n \alpha_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n \sum_{j\in N(i)}1(y_i=y_j)\right)\\\\ &=C_0\cdot\text{exp}\left(\sum_{i=1}^n \text{log}\left(f_{Y_i}(\cdot)\right)\right)\cdot C\cdot\text{exp}\left(\sum_{i=1}^n \alpha_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n \sum_{j\in N(i)}1(y_i=y_j)\right)\\\\ &=\tilde{C}\cdot\text{exp}\left(\sum_{i=1}^n\left(\alpha_{i,y_i}+\text{log}\left(f_{Y_i}(\cdot)\right)\right)+\frac{1}{2}\beta\sum_{i=1}^n\sum_{j\in N(i)}1(y_i=y_j)\right)\\\\ &=\tilde{C}\cdot\text{exp}\left(\sum_{i=1}^n\tilde\alpha_{i,y_i}+\frac{1}{2}\beta\sum_{i=1}^n\sum_{j\in N(i)}1(y_i=y_j)\right) \end{align*}$$

for appropriate choices of $\tilde\alpha_{i,k}$ and a new normalizing constant $\tilde{C}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.