Can you write a Geometric random variable as some combination of Bernoulli random variables?

Background

Given $$Y \sim \text{Binomial(n,p)}$$, we can write $$Y = \sum_{i=1}^{n} X_i$$ where $$X_1,X_2,...,X_n$$ are iid $$\text{Bernoulli}(p)$$. This is useful in, for example, determining the mean of a binomial random variable: $$E(Y)=E\left(\sum X_i\right) = \sum E(X_i) = np$$

Question

If we are given $$Y \sim \text{Geometric(p)}$$, can we similarly write $$Y$$ as some combination of Bernoulli random variables?

• It would have to be an infinite combination, wouldn't it? By "combination" do you mean a sum of independent variables or would you include more general operations? If so, what would they be?
– whuber
Dec 7, 2020 at 20:14
• I was trying to think about how to do it with a sum, but I couldn't make it work. So I decided to put "combination" instead. Yes, I would include more general operations. I'm not sure what they would be... I was thinking we need some way to mathematically "keep track" of whether we have gotten a success, because once we get a success, we stop the Bernoulli trials. But I'm not sure what that would look like. I think if we did have some kind of infinite sequence of operations and the "keep track" operation automatically becomes 0 once we get a success, it might work? Dec 7, 2020 at 20:19
• Trivially, every discrete distribution is a mixture of (shifted or scaled) Bernoulli distributions. A less trivial result is that the geometric distribution cannot be expressed as the sum of independent Bernoulli distributions, as you have already figured out.
– whuber
Dec 7, 2020 at 20:25

For clarity, I am going to look at the version of the geometric distribution with support on the non-negative integers, with expected value $$\mathbb{E}(Y) = (1-p)/p$$. Now, suppose we have a sequence of Bernoulli random variables $$X_1,X_2,X_3,... \sim \text{IID Bern}(p)$$ to use for the construction.

The geometric random variable $$Y$$ can be interpreted as the number of "failures" that occur before the first "success", so it can be written as:

\begin{align} Y &\equiv \max \ \{ y = 0,1,2,... | X_1 = \cdots = X_{y} = 0 \} \\[12pt] &= \max \Bigg\{ y = 0,1,2,... \Bigg| \prod_{\ell = 1}^{y} (1-X_\ell) = 1 \Bigg\} \\[6pt] &= \sum_{i=1}^\infty \prod_{\ell = 1}^{i} (1-X_\ell). \\[6pt] \end{align}

This is probably the "simplest" you can write the expression, since it accords with the descriptive intuition of what the random variable represents. As whuber notes in the comments, every discrete distribution can be written as a mixture of a countable number of shifted or scaled Bernoulli random variables (and indeed, there are an infinite number of ways to do this).

(Note: For the other version of the geometric distribution, with support on the positive integers, the random variable can be interpreted as the number of trials that occur by the time of the first "success", which is one more than the number of "failures". In this case you just add one to the above expression to get construct the random variable.)

Confirming the result: To see that this expression is adequate, note that:

\begin{align} F_Y(y) \equiv \mathbb{P}(Y \leqslant y) &= 1 - \mathbb{P}(Y \geqslant y+1) \\[12pt] &= 1 - \mathbb{P} \Bigg( \prod_{\ell = 1}^{y+1} (1-X_\ell) = 1 \Bigg) \\[6pt] &= 1 - \prod_{\ell = 1}^{y+1} \mathbb{P}(1-X_\ell = 1) \\[6pt] &= 1 - \prod_{\ell = 1}^{y+1} \mathbb{P}(X_\ell = 0) \\[6pt] &= 1 - \prod_{\ell = 1}^{y+1} (1-p) \\[6pt] &= 1 - (1-p)^{y+1}, \\[6pt] \end{align}

which is the CDF of the (chosen version of the) geometric distribution.

• This is cool! It looks like we can compute $E(Y)$ using your result, although I'm not sure about one step. $$E(Y) = E\left(\sum_{i=0}^{\infty} \prod_{l=0}^{i} (1-X_i)\right)=\sum_{i=0}^{\infty} \prod_{l=0}^{i}E(1-X_i)=\sum_{i=0}^{\infty}(1-p)^i=\frac{1-p}{p}$$ We can move the expectation inside the product because of independence. The only thing I'm not sure about is moving the expectation inside an infinite sum. I think it's justified because (with probability 1) the tail terms of the sum are 0. Dec 7, 2020 at 22:56
• You can move an expectation inside the infinite sum via the linearity property. You can then move it inside the product via independence. There are some errors in your working (e.g., wrong subscripts, etc). It should be: $$\mathbb{E}(Y) = \mathbb{E} \Bigg( \sum_{i=1}^\infty \prod_{\ell = 1}^{i} (1-X_\ell) \Bigg) = \sum_{i=1}^\infty \prod_{\ell = 1}^{i} \mathbb{E} (1-X_\ell) = \sum_{i=1}^\infty (1-p)^i = \frac{1-p}{1-(1-p)} = \frac{1-p}{p}.$$
– Ben
Dec 7, 2020 at 23:01
• Yes, I see that now. Thanks! Dec 7, 2020 at 23:06