12
$\begingroup$

While studying probability, I am kind of having difficulties in understanding marginal probability distribution and conditional probability distribution. To me, they look much the same and cannot find the clear concepts of differences in those two probability distributions.

$\endgroup$
1
  • $\begingroup$ was my answer being best revoked because it no longer answers the question? $\endgroup$ – develarist Sep 17 '20 at 0:55
16
$\begingroup$

Let me add an example to @develarist's answer. \begin{array}{llcc|r} Y & & y_1 & y_2 \\ \hline X & x_1 & 0.450 & 0.150 & 0.600 \\ & x_2 & 0.167 & 0.233 & 0.400 \\ \hline & & 0.617 & 0.383 & 1.000 \end{array}

The table shows the joint distribution of $(X,Y)$: \begin{array}{l} P(X=x_1,Y=y_1)=0.450 \\ P(X=x_1,Y=y_2)=0.150 \\ P(X=x_2,Y=y_1)=0.167 \\ P(X=x_2,Y=y_2)=0.233 \\ \end{array}

The marginal distribution of $Y$ is: \begin{align*} P(Y=y_1)&=P(Y=y_1 \text{ and } (X=x_1\text{ or }X=x_2))\\ &= P((Y=y_1\text{ and }X=x_1)\text{ or }(Y=y_1\text{ and }X=x_2)) \\ &= \sum_{i=1}^2 P(Y=y_1,X=x_i)=0.450+0.167=0.617 \\ P(Y=y_2)&=0.383 \end{align*} The marginal distribution of $X$ is: \begin{align*} P(X=x_1)&=0.600\\ P(X=x_2)&=0.400 \end{align*}

The conditional distribution of $Y$ given $X=x_1$ is: \begin{align*} P(Y=y_1\mid X=x_1)&=\frac{P(Y=y_1,X=x_1)}{P(X=x_1)}\\&=0.450/0.600=0.750\\ P(Y=y_2\mid X=x_1)&=0.150/0.600=0.250 \end{align*}

The conditional distribution of $Y$ given $X=x_2$ is: \begin{array}{l} P(Y=y_1\mid X=x_2)=0.167/0.400=0.4175\\ P(Y=y_2\mid X=x_2)=0.233/0.400=0.5825 \end{array}

The conditional distribution of $X$ given $Y=y_1$ is: \begin{array}{l} P(X=x_1\mid Y=y_1)=0.450/0.617=0.7293\\ P(X=x_2\mid Y=y_1)=0.167/0.617=0.2707 \end{array}

The conditional distribution of $X$ given $Y=y_2$ is: \begin{array}{l} P(X=x_1\mid Y=y_2)=0.150/0.383=0.3916\\ P(X=x_2\mid Y=y_2)=0.233/0.383=0.6084 \end{array}

$\endgroup$
1
  • 6
    $\begingroup$ Notice how the Marginal Distributions, i.e. [0.617, 0.383] and [0.600,0.400] are in the margins (i.e at the edges or on the sides) of the first table above, hence the word "marginal" $\endgroup$ – noob2 Aug 30 '20 at 15:42
13
$\begingroup$

If $X$ and $Y$ are two random variables, the univariate pdf of $X$ is the marginal distribution of $X$, and the univariate pdf of $Y$ is the marginal distribution of $Y$. Therefore, when you see the word marginal, just think of a single data series' own distribution, itself. don't be tricked into thinking marginal means something different or special than a univariate (single variable) assessment.

For conditional distribution on the other hand, we make a bivariate (two variables) assessment, but by considering the univariate components' relationship to one another: conditional pdf is the distribution of $X$ conditional on, or given the recognition of, $Y$'s data. The idea is that an observation in $X$ has some correspondence to a similarly-located observation in $Y$, and therefore we're thinking of $X$ with respect to what is observed in $Y$. In other words, the conditional pdf is a flimsy way of characterizing the distribution of $X$ as a function of $Y$.

$\endgroup$
2
  • 10
    $\begingroup$ "don't be tricked into thinking marginal means something different or special than a univariate (single variable) assessment": a marginal distribution doesn't have to be univariate; that's only the case when you start with 2 variables. If you have a joint distribution of X,Y,Z, then the distribution of (X, Y) is also a marginal probability distribution. Marginal just means it has to be a smaller subset of variables than the full joint distribution. $\endgroup$ – Scott Staniewicz Aug 29 '20 at 18:05
  • 5
    $\begingroup$ to make the answer less broad for understanding, my example starts with "two random variables" only $\endgroup$ – develarist Aug 29 '20 at 18:08
4
$\begingroup$

In this question/answer I used the following graph:

showcase

  • Joint distribution In the left plot you see the joint distribution of disp versus mpg. This is a scatterplot in a 2D-space.

  • Marginal distribution You might be interested in the distribution of all the 'mpg' together. That is depicted by the first (big) histogram. It shows the distribution of 'mpg'. (note that in this way of plotting the marginal distribution occurs in the margins of the figure)

  • Conditional distribution can be seen as slices through the scatter plot. In this case you see the distribution of the variable 'mpg' for three different conditions (emphasized in the histogram and joint distribution with colors yellow, green and blue).

$\endgroup$
4
  • $\begingroup$ how was the yellow, green and blue conditional distributions estimated in the first scatter plot, relative to the underlying full sample joint distribution. and where can I see similar visualizations of the conditional distribution $\endgroup$ – develarist Sep 17 '20 at 0:36
  • $\begingroup$ @develarist the conditions are respectively 'disp <= 100', '100<disp<=300' and '300<disp<=500'. $\endgroup$ – Sextus Empiricus Sep 17 '20 at 0:38
  • $\begingroup$ know of any conditional distribution scatter plots for (conditionally) probabilistic examples rather than range conditions $\endgroup$ – develarist Sep 17 '20 at 0:41
  • $\begingroup$ So these are wide bands as conditions. Often we see equalities as the condition, but that is difficult to express for such coarse data. A similar visualization is repeated here and in that visualization the bands are sharper (but in that visualization I made a mistake with the marginal distribution, which is not a sum of the three conditional distributions. and now I also see that I made a mistake with the label X~100 which should be X~200). $\endgroup$ – Sextus Empiricus Sep 17 '20 at 0:41
2
$\begingroup$

In general, the joint distribution of two or more variables $P(A, B, C, ...)$ is a statement of what you know, assuming you have no certain information about any of them (i.e., you are uncertain about all of them).

Given a joint distribution, it's often relevant to look at subsets of the variables. The distribution of a subset, ignoring any others, is called a marginal distribution. For example, $P(A)$ is a marginal distribution of $P(A, B, C, ....)$, $P(A, C)$ is also a marginal distribution of $P(A, B, C, ...)$, likewise $P(B), P(B, Z), P(H, W, Y), P(C, E, H, Z)$, etc., are all marginal distributions of $P(A, B, C, ...)$.

By "ignoring", I mean that the omitted variables could take on any values; we don't make any assumption about them. A different way to look at subsets is to make assumptions about the omitted variables. That is, to look at some of the variables assuming we know something about the others. This is called a conditional distribution and it is written with a vertical bar to separate the uncertain variables, on the left, from the assumed variables, on the right.

For example, $P(B, C | A), P(D | A, J, X), P(C, M | O, Q, R, U), P(D, F, G, L | B, E, S)$, etc., are all conditional distributions derived from the joint distribution $P(A, B, C, ...)$. These all represent statements of the form: given that we know the variables on the right, what do we know about the uncertain variables on the left. E.g., $P(B, C | A)$ represents what we know about $B$ and $C$, given that we know $A$. Likewise $P(D | A, J, X)$ represents what we know about $D$, given that we know $A, J$, and $X$.

There may be any numbers of variables on the left and right in a conditional distribution. $P(C, M | O, Q, R, U)$ represents what we know about $C$ and $M$, given that we know $O, Q, R$, and $U$. $P(D, F, G, L | B, E, S)$ represents what we know about $D, F, G,$ and $L$, given that we know $B, E,$ and $S$.

Joint, marginal, and conditional distributions are related in some important ways. In particular, $P($some variables, other variables$) = P($some variables $|$ other variables$) P($other variables$)$. That is, the joint distribution of some variables and other variables is the product of the conditional distribution of some variables given other variables and the marginal distribution of other variables. This is a generalization of Bayes' rule.

$\endgroup$
2
$\begingroup$

Consider a joint discrete probability $p(x_i,y_j)$ over $x_i$'s and $y_j$'s. The marginal probability $p_X(x_i)$ has no dependence on any $Y$ any more since we sum over all $y_j$ as follows: $p_X(x_i) = \sum_j p(x_i, y_j)$. We've reduced the two-dimensional information from $p(x_i,y_j)$ into one dimension $p_X(x_i)$.

The conditional distribution of $X$ conditioned on $Y$ is a distribution of $X$, given a specific value of $Y$, using conditional probability defined as $p(X=x_i \mid Y=y_j)$ and looking at all values of $X$. So for every value of $Y$, we have a different conditional distribution for $X$ conditioned on that value of $Y$.

$\endgroup$
2
  • $\begingroup$ What do you mean by "no dependence on any $Y$ since we sum over all $y_j$"? What does the meaning of dependence have to do with summing? $\endgroup$ – StoryMay Aug 31 '20 at 5:37
  • 1
    $\begingroup$ Dependence in this case is informal language. Basically the marginal probability $p_X(x_i)$ does not have any $y$ appearing anywhere any more. Now it's just a function of values of $X$. $\endgroup$ – qwr Aug 31 '20 at 5:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.