# How can I calculate a joint distribution based on marginal and conditional information?

Suppose I am given a probability mass function. A second one is also specified conditional on the first. Together, they must define a joint probability distribution. How can I calculate it?

• I find that visualizing the joint distribution with diagrams like the first one at stats.stackexchange.com/a/104018/919 can be helpful in working through problems like this. You can construct such a diagram from the conditional distribution: in effect, it tells you how to draw each column of dots on $(n,k)$ axes, and then finding the marginal distribution of $K$ is matter of adding up the rows of probabilities.
– whuber
Mar 2, 2020 at 15:31
• Calculating the marginal PMF from the Joint PMF by using the tabular method. See nice table produced above in this string. We know that adding rows or columns to get the Marginal PMF. Rows PK (k) and Columns PN(n). I am not quite sure, but I believe Marginal PMF of PN(n) = Summation 1/2 * n * [2^(-k)] /2n Marginal PMF of PK (k) = Summation 1/2 * n * 2 * [2^(-2k+1)]/2n Feel free to find my errors or show that I have correctly interpreted the graphical output above. Oct 5, 2020 at 4:26

Sketching a diagram of the joint distribution might firm up your understanding as well as help you get the right answer (and spot incorrect answers that might be offered).

Usually you don't need to work very hard at this--it's primarily a conceptual exercise--but for accuracy I asked a computer to draw this diagram: This diagram situates a dot at each point with coordinates $$(n,k)$$ where the probability is nonzero. (The dots are colored and sized in proportion to their probabilities.) To do that, it creates vertical strips of dots, because each such strip corresponds to an event with a single value of $$n.$$ Those strips therefore reflect the conditional probability information, which says the strip positioned over a whole number $$n$$ must have dots at heights $$k=1,2,\ldots, 2n$$ (all with equal probabilities within each strip). I have highlighted the strip for $$n=4$$ because that corresponds to the work attempted in the question.

To the right of each dot I have posted a formula for the joint (not the conditional) probability. Recall that the joint probability at $$(n,k)$$ must be the product of

• The probability of $$n,$$ given by $$1/2\times n \times 2^{-n},$$ and

• The conditional probability of $$k$$ given $$n.$$ Because this is assumed uniform and it covers $$2n$$ possibilities, this conditional probability is $$1/(2n).$$

Thus the formula for the joint probability is

$$P(n,k) = P(n)\times P(k\mid n) = \left\{\begin{array}{lr}\frac{1}{2}\,n\,2^{-n}\,/\,(2n) & 1 \le k \le 2n \\ 0 & \text{otherwise.}\end{array}\right.$$

Notice that the expression on the right hand side simplifies:

$$\frac{1}{2}\,n\,2^{-n}\,/\,(2n) = 2^{-(n+2)}.$$

You can spot-check these values against the diagram if you wish.

As a reality check, let's verify the probabilities sum to unity--but we'll do this in a different way than we constructed the diagram so that the check might catch any mistakes we might have made. Let's sum the probabilities by rows:

• At the bottom two rows with $$k=1$$ and $$k=2,$$ which are identical, you can read off the sequence of probabilities left to right as $$2^{-1-2}, 2^{-2-2}, 2^{-3-2}, \ldots = 1/8, 1/16, 1/32, \ldots.$$ This is a geometric series that sums (obviously) to $$1/4.$$ The two rows together sum to $$1/2.$$

• At the next two rows with $$k=3$$ and $$k=4,$$ which are identical, the sequence of probabilities is the same as before, but with the first one omitted. We obtain two rows summing to $$1/16 + 1/32 + 1/64 + \cdots = 1/8.$$ The two rows together sum to $$1/4.$$

• The pattern is evident: every time you go up two rows you see the same probabilities as before but (a) multiplied by $$1/2$$ and (b) shifted one unit to the right. Thus the next two sum to $$1/4\times 1/2 = 1/8,$$ the next two sum to $$1/8\times 1/2=1/16,$$ and so on.

Evidently the sum of all the probabilities is $$1/2 + 1/4 + 1/8 + \cdots = 1,$$ as it should be.

As a mathematical proposition, this diagram has shown how to evaluate the sum

$$\sum_{n=1}^\infty n\, 2^{-n} = 2$$

by splitting each term $$n 2^{-n}$$ into $$2n$$ separate pieces of size $$2^{-(n+1)}$$ and then adding those pieces in a different order. The evaluation requires knowing only that $$1/2+1/4+1/8+\cdots = 1.$$

• I forgot to mention it but I thought this answer was brilliant. The diagram you made does drive the point home. Thank you. Mar 10, 2020 at 14:07

The conditional distribution is $$p_{K|N=n}(k)=\frac{1}{2n},\ \ \ k=1,\dots,2n$$

For joint, you just multiply this with $$p_N(n)$$, which you already did with slightly wrong conditional, i.e. $$1/n$$. You don't necessarily have $$k$$ as a variable in your joint distribution. As far as uniform distribution is concerned, disappearance of some variables from PMF/PDF shouldn't surprise you. So, the joint is: $$p_{N,K}(n,k)=\frac{2^{-n}}{4},\ \ n\in \mathbb{Z^+}, k\in [1,2n], k\in \mathbb{Z^+}$$

• I think you might have overlooked the factor of $n$ in the formula for the distribution of $N.$
– whuber
Mar 2, 2020 at 16:24
• @whuber oh, yes! Mar 2, 2020 at 16:57