What are the differences between "Marginal Probability Distribution" and "Conditional Probability Distribution"?

Question

While studying probability, I am kind of having difficulties in understanding marginal probability distribution and conditional probability distribution. To me, they look much the same and cannot find the clear concepts of differences in those two probability distributions.

Answer 1

Let me add an example to @develarist's answer. \begin{array}{llcc|r} Y & & y_1 & y_2 \\ \hline X & x_1 & 0.450 & 0.150 & 0.600 \\ & x_2 & 0.167 & 0.233 & 0.400 \\ \hline & & 0.617 & 0.383 & 1.000 \end{array}

The table shows the joint distribution of $(X,Y)$: \begin{array}{l} P(X=x_1,Y=y_1)=0.450 \\ P(X=x_1,Y=y_2)=0.150 \\ P(X=x_2,Y=y_1)=0.167 \\ P(X=x_2,Y=y_2)=0.233 \\ \end{array}

The marginal distribution of $Y$ is: \begin{align*} P(Y=y_1)&=P(Y=y_1 \text{ and } (X=x_1\text{ or }X=x_2))\\ &= P((Y=y_1\text{ and }X=x_1)\text{ or }(Y=y_1\text{ and }X=x_2)) \\ &= \sum_{i=1}^2 P(Y=y_1,X=x_i)=0.450+0.167=0.617 \\ P(Y=y_2)&=0.383 \end{align*} The marginal distribution of $X$ is: \begin{align*} P(X=x_1)&=0.600\\ P(X=x_2)&=0.400 \end{align*}

The conditional distribution of $Y$ given $X=x_1$ is: \begin{align*} P(Y=y_1\mid X=x_1)&=\frac{P(Y=y_1,X=x_1)}{P(X=x_1)}\\&=0.450/0.600=0.750\\ P(Y=y_2\mid X=x_1)&=0.150/0.600=0.250 \end{align*}

The conditional distribution of $Y$ given $X=x_2$ is: \begin{array}{l} P(Y=y_1\mid X=x_2)=0.167/0.400=0.4175\\ P(Y=y_2\mid X=x_2)=0.233/0.400=0.5825 \end{array}

The conditional distribution of $X$ given $Y=y_1$ is: \begin{array}{l} P(X=x_1\mid Y=y_1)=0.450/0.617=0.7293\\ P(X=x_2\mid Y=y_1)=0.167/0.617=0.2707 \end{array}

The conditional distribution of $X$ given $Y=y_2$ is: \begin{array}{l} P(X=x_1\mid Y=y_2)=0.150/0.383=0.3916\\ P(X=x_2\mid Y=y_2)=0.233/0.383=0.6084 \end{array}

Answer 2

If $X$ and $Y$ are two random variables, the univariate pdf of $X$ is the marginal distribution of $X$, and the univariate pdf of $Y$ is the marginal distribution of $Y$. Therefore, when you see the word marginal, just think of a single data series' own distribution, itself. don't be tricked into thinking marginal means something different or special than a univariate (single variable) assessment.

For conditional distribution on the other hand, we make a bivariate (two variables) assessment, but by considering the univariate components' relationship to one another: conditional pdf is the distribution of $X$ conditional on, or given the recognition of, $Y$'s data. The idea is that an observation in $X$ has some correspondence to a similarly-located observation in $Y$, and therefore we're thinking of $X$ with respect to what is observed in $Y$. In other words, the conditional pdf is a flimsy way of characterizing the distribution of $X$ as a function of $Y$.

Answer 3

In this question/answer I used the following graph:

• Joint distribution In the left plot you see the joint distribution of disp versus mpg. This is a scatterplot in a 2D-space.

• Marginal distribution You might be interested in the distribution of all the 'mpg' together. That is depicted by the first (big) histogram. It shows the distribution of 'mpg'. (note that in this way of plotting the marginal distribution occurs in the margins of the figure)

• Conditional distribution can be seen as slices through the scatter plot. In this case you see the distribution of the variable 'mpg' for three different conditions (emphasized in the histogram and joint distribution with colors yellow, green and blue).

Answer 4

In general, the joint distribution of two or more variables $P(A, B, C, ...)$ is a statement of what you know, assuming you have no certain information about any of them (i.e., you are uncertain about all of them).

Given a joint distribution, it's often relevant to look at subsets of the variables. The distribution of a subset, ignoring any others, is called a marginal distribution. For example, $P(A)$ is a marginal distribution of $P(A, B, C, ....)$, $P(A, C)$ is also a marginal distribution of $P(A, B, C, ...)$, likewise $P(B), P(B, Z), P(H, W, Y), P(C, E, H, Z)$, etc., are all marginal distributions of $P(A, B, C, ...)$.

By "ignoring", I mean that the omitted variables could take on any values; we don't make any assumption about them. A different way to look at subsets is to make assumptions about the omitted variables. That is, to look at some of the variables assuming we know something about the others. This is called a conditional distribution and it is written with a vertical bar to separate the uncertain variables, on the left, from the assumed variables, on the right.

For example, $P(B, C | A), P(D | A, J, X), P(C, M | O, Q, R, U), P(D, F, G, L | B, E, S)$, etc., are all conditional distributions derived from the joint distribution $P(A, B, C, ...)$. These all represent statements of the form: given that we know the variables on the right, what do we know about the uncertain variables on the left. E.g., $P(B, C | A)$ represents what we know about $B$ and $C$, given that we know $A$. Likewise $P(D | A, J, X)$ represents what we know about $D$, given that we know $A, J$, and $X$.

There may be any numbers of variables on the left and right in a conditional distribution. $P(C, M | O, Q, R, U)$ represents what we know about $C$ and $M$, given that we know $O, Q, R$, and $U$. $P(D, F, G, L | B, E, S)$ represents what we know about $D, F, G,$ and $L$, given that we know $B, E,$ and $S$.

Joint, marginal, and conditional distributions are related in some important ways. In particular, $P($some variables, other variables$) = P($some variables $|$ other variables$) P($other variables$)$. That is, the joint distribution of some variables and other variables is the product of the conditional distribution of some variables given other variables and the marginal distribution of other variables. This is a generalization of Bayes' rule.

Answer 5

Consider a joint discrete probability $p(x_i,y_j)$ over $x_i$'s and $y_j$'s. The marginal probability $p_X(x_i)$ has no dependence on any $Y$ any more since we sum over all $y_j$ as follows: $p_X(x_i) = \sum_j p(x_i, y_j)$. We've reduced the two-dimensional information from $p(x_i,y_j)$ into one dimension $p_X(x_i)$.

The conditional distribution of $X$ conditioned on $Y$ is a distribution of $X$, given a specific value of $Y$, using conditional probability defined as $p(X=x_i \mid Y=y_j)$ and looking at all values of $X$. So for every value of $Y$, we have a different conditional distribution for $X$ conditioned on that value of $Y$.