How can I calculate a joint distribution based on marginal and conditional information?

Question

Suppose I am given a probability mass function. A second one is also specified conditional on the first. Together, they must define a joint probability distribution. How can I calculate it?

Answer 1

Sketching a diagram of the joint distribution might firm up your understanding as well as help you get the right answer (and spot incorrect answers that might be offered).

Usually you don't need to work very hard at this--it's primarily a conceptual exercise--but for accuracy I asked a computer to draw this diagram:

This diagram situates a dot at each point with coordinates $(n,k)$ where the probability is nonzero. (The dots are colored and sized in proportion to their probabilities.) To do that, it creates vertical strips of dots, because each such strip corresponds to an event with a single value of $n.$ Those strips therefore reflect the conditional probability information, which says the strip positioned over a whole number $n$ must have dots at heights $k=1,2,\ldots, 2n$ (all with equal probabilities within each strip). I have highlighted the strip for $n=4$ because that corresponds to the work attempted in the question.

To the right of each dot I have posted a formula for the joint (not the conditional) probability. Recall that the joint probability at $(n,k)$ must be the product of

• The probability of $n,$ given by $1/2\times n \times 2^{-n},$ and

• The conditional probability of $k$ given $n.$ Because this is assumed uniform and it covers $2n$ possibilities, this conditional probability is $1/(2n).$

Thus the formula for the joint probability is

$$P(n,k) = P(n)\times P(k\mid n) = \left\{\begin{array}{lr}\frac{1}{2}\,n\,2^{-n}\,/\,(2n) & 1 \le k \le 2n \\ 0 & \text{otherwise.}\end{array}\right.$$

Notice that the expression on the right hand side simplifies:

$$\frac{1}{2}\,n\,2^{-n}\,/\,(2n) = 2^{-(n+2)}.$$

You can spot-check these values against the diagram if you wish.

---

As a reality check, let's verify the probabilities sum to unity--but we'll do this in a different way than we constructed the diagram so that the check might catch any mistakes we might have made. Let's sum the probabilities by rows:

• At the bottom two rows with $k=1$ and $k=2,$ which are identical, you can read off the sequence of probabilities left to right as $2^{-1-2}, 2^{-2-2}, 2^{-3-2}, \ldots = 1/8, 1/16, 1/32, \ldots.$ This is a geometric series that sums (obviously) to $1/4.$ The two rows together sum to $1/2.$

• At the next two rows with $k=3$ and $k=4,$ which are identical, the sequence of probabilities is the same as before, but with the first one omitted. We obtain two rows summing to $1/16 + 1/32 + 1/64 + \cdots = 1/8.$ The two rows together sum to $1/4.$

• The pattern is evident: every time you go up two rows you see the same probabilities as before but (a) multiplied by $1/2$ and (b) shifted one unit to the right. Thus the next two sum to $1/4\times 1/2 = 1/8,$ the next two sum to $1/8\times 1/2=1/16,$ and so on.

Evidently the sum of all the probabilities is $1/2 + 1/4 + 1/8 + \cdots = 1,$ as it should be.

---

As a mathematical proposition, this diagram has shown how to evaluate the sum

$$\sum_{n=1}^\infty n\, 2^{-n} = 2$$

by splitting each term $n 2^{-n}$ into $2n$ separate pieces of size $2^{-(n+1)}$ and then adding those pieces in a different order. The evaluation requires knowing only that $1/2+1/4+1/8+\cdots = 1.$

Answer 2

The conditional distribution is $$p_{K|N=n}(k)=\frac{1}{2n},\ \ \ k=1,\dots,2n$$

For joint, you just multiply this with $p_N(n)$, which you already did with slightly wrong conditional, i.e. $1/n$. You don't necessarily have $k$ as a variable in your joint distribution. As far as uniform distribution is concerned, disappearance of some variables from PMF/PDF shouldn't surprise you. So, the joint is: $$p_{N,K}(n,k)=\frac{2^{-n}}{4},\ \ n\in \mathbb{Z^+}, k\in [1,2n], k\in \mathbb{Z^+}$$