In message-passing methods, what is the actual content of the messages? In message-passing methods, factors and random variables exchange messages that typically encode marginals, but as much as I look at their formulas, I still don't understand what those messages actually look like.
For example, in belief propagation, when using the factor graph representation, we have the following scheme:

*

*Messages from a variable node $v$ to a factor node $u$
$\forall x_v\in Dom(v),\; \mu_{v \to u} (x_v) = \prod_{u^* \in N(v)\setminus\{u\} } \mu_{u^* \to v} (x_v).$


*Messages from a factor node $u$ to a variable node $v$ are the product of the factor with messages from all other nodes, marginalised over all variables except the one associated with $v$:
$\forall x_v\in Dom(v),\; \mu_{u \to v} (x_v) = \sum_{\mathbf{x}'_u:x'_v = x_v } f_u (\mathbf{x}'_u) \prod_{v^* \in N(u) \setminus \{v\}} \mu_{v^* \to u} (x'_{v^*}).$
I understand that these messages represent marginals and that the eventually we end up with the marginals in every node/factor, but my question is:

*

*Are these messages  vectors holding real numbers?

*What is the dimensionality of those vectors?

*What does this dimensionality represent?

To make things more concrete let's assume a distribution over real random variables (e.g. gaussian), although I am also interested in the discrete case (e.g. categorical random variables).  Also, how are these messages initialized?
 A: This is a problem I am currently working on so I'm glad to share my current understanding about it. The messages are essentially computed by composing probability density functions over the variables. Remember that in a factor graph each edge has always a factor on one end and a variable on the other, so the message associated with the edge is a distribution on the variable incident on that edge. The formulas that you wrote are the same formulas found in most textbooks and they will specialize differently depending on whether you have continuous or discrete distributions. In the case of the continuous Normal distribution, for example, one can work out the products and the sums (integrals) and find closed form expressions for the messages. Note that in this case what you are propagating are actually the parameters of the distributions ($\mu$,$\sigma^2$). If you can't find closed for expressions for the messages than you have to revert to discrete distributions. As you are suggesting, in this case you are passing vectors, that are essentially histograms. So the dimension of these vectors is associated with the number of quantization intervals of the histograms.
