I am studying about Gibbs distribution concept and I am confusing a one term in that concept that is normalization term. According to the Hammersley–Clifford theorem, an random $x$ can equivalently be characterized by a Gibbs distribution $$P(x)=Z^{-1}\exp(-U(x))$$ where $Z$ is a normalizing constant called the partition function $$Z=\sum_x \exp(-U(x))$$ and $U(x)$ is an energy function of the form $$U(x)=\sum_{c \in C}V_c(x)$$ which is a sum of clique potentials $V_c(x)$ over all possible cliques $C$. A clique $c$ is defined as a subset of sites in $S$ (are neighborhood system of $x$) in which every pair of distinct sites are neighbors, except for single-site cliques

To make clearly understand, let get simple example. Assume $x$ has two values {0,1}. The $U(x=0)$ can be computed as

$$ \left[ \begin{array}{cc|c} 0&8&-8\\ 0&-8&0\\ 3&-4&5 \end{array} \right] $$ $U(x=1)$ is $$ \left[ \begin{array}{cc|c} 0&-8&8\\ 4&-5&-8\\ 7&8&-9 \end{array} \right] $$

My question is that how to compute $Z$ and $P(x=0)$ as well as $P(x=1)$?

As my understand, $Z$ is normalization over all $x$. Then $Z$ is computed $$Z=\sum_x \exp(-U(x))=\exp(-U(x=0))+\exp(-U(x=1))$$ $$Z=10^3 \times \left[ \begin{array}{cc|c} 0.002&2.9810&2.981\\ 0.001&3.1294&2.9820\\ 0.0001&0.0546&8.1031 \end{array} \right]$$

Then $P(x=0)$ and $P(x=1)$ are computed $$P(x=0)=\frac {\exp(-U(x=0))}{Z}=\left[ \begin{array}{cc|c} 0.5&00&1\\ 0.9820&0.9526&0.0003\\ 0.9820&1&0 \end{array} \right]$$

$$P(x=1)=\frac {\exp(-U(x=1))}{Z}=\left[ \begin{array}{cc|c} 0.5&1&0\\ 0.018&0.0474&0.9997\\ 0.0180&0&1 \end{array} \right]$$

Is it correct? Reference document in here

  • $\begingroup$ The link is broken and I tried and fond that I cannot help you fix it. $\endgroup$ – Lerner Zhang Apr 6 '19 at 2:47

I don't know what you meant by the two tables of the bivariate $U$, I thought maybe those two tables are the marginal (unnormalized) distributions of two variables. If so, I thought you don't need to do any operation over the variables(left two columns) you just need to calculate the potentials(right column). And for more information please read this concrete example in this slide.

To answer your question directly, the normalization term is the Z in the slide which is the sum of all possible potentials(multiplication of a set of assignments) of the variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.