I am quite new to Probabilistic Graphical Model (PGMs)

As I am reading about it, the theory starts with a set of random variables and its joint probability distribution. But nobody talks about underlying common sample space over which these variables are defined and nature of experiment we are performing. It may not matter as far as using PGMs to solve problems are concerned (or perhaps it does!!).

The question is as follows. I don't even know if it makes sense.

  1. What is underlying common sample space for these random variables.

PGMs provide a framework for reasoning about multivariate probability distributions in a manner that is both conceptually and computationally convenient. Your question, however, is not intrinsically tied to PGMs, but rather about multivariate distributions in general. Since PGMs are an abstract framework, there is no need to explicitly refer to any particular underlying sample space in developing their theory.

Nevertheless, here is an example illustrating the general concept of a random vector. Consider the experiment of flipping two coins and rolling a die. We can define the sample space to be the set of ordered triples

$$\Omega = \{(c_1, c_2, d): c_1, c_2 \in \{0,1\}, d \in \{1,\dots, 6\}\}, $$

where 1 represents flipping a head for $c_1$ and $c_2$. Let $X$ be the random variable denoting the number of heads flipped and let $Y$ be the random variable denoting the outcome of the die roll. Then $X$ and $Y$ are both functions that assign a non-negative integer to each outcome in $\Omega$. For example, $X(1,1,3) = 2$ and $Y(1,1,3) = 3$. If we assume that the two flips and the roll are independent and that the coin and the die are fair, then the multivariate distribution over the random vector $(X,Y)$ would be such that $P(X = 2, Y = 3) = 1/24$. The graphical model for this system would be fairly boring: it would consist of two nodes (one for each r.v.) and no edges.

If we don't assume the coin is fair, we can model the probability of heads $\theta$ as a random variable itself, which (assuming we're working with a Bayesian network) would be its own node with a directed edge into the node for $X$.

  • $\begingroup$ So we are taking cartesian product of individual sample spaces. $\endgroup$
    – user634615
    Jul 4 '17 at 7:41
  • $\begingroup$ Yes, in general when you have a probability experiment that consists of several stages, the sample space for the whole experiment is the product of the sample spaces for each of the stages. $\endgroup$
    – tddevlin
    Jul 5 '17 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.