Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables

Question

Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$.

If $S_n = X_1 + X_2 + \ldots + X_n$ and $N = \inf\{n \geq 1: S_n > 1\}$, then what would be the value of $\mathrm{Var}(N)$ ?

Answer 1

I am going to use the definition $N=\inf \{ n\ge0: S_{n+1}> 1 \}$ which seems more natural.

The event $\{N\le n\}$ is the same as $\{S_{n+1} > 1\}$ (or equivalently, $\{N>n\}$ is the same as $\{S_{n+1} \le 1\}$).

By the tail-sum formula, we can thus get the expectation of $N$ as \[ E[N] = \sum_{n=0}^\infty P(N>n) = \sum_{n=0}^\infty P(S_{n+1}\le 1). \] Now for any fixed $n$, $S_n$ has the $Gamma(n,1/\lambda)$ distribution (because it's a sum of exponential random variables) and by looking up the density of such distribution, we get \[ P(S_{n+1} \le 1) = \int_0^1 \frac{1}{\Gamma(n+1)}\lambda^{n+1}x^{n}e^{-x\lambda}dx = \lambda \int_0^1 \frac{1}{n!}(x\lambda)^{n}e^{-x\lambda}dx \] Since $\exp(x\lambda)=\sum_{n=0}^\infty (x\lambda)^{n}/n!$, we obtain by moving the integral outside of sum by Fubini's theorem \[ E[N] = \lambda \int_0^1 \exp(x\lambda) e^{-x\lambda} dx = \lambda. \]

We can compute $E[N^2]$ similarly using the tail formula, by making "packets" \[ E[N^2] = \sum_{n=0}^\infty P(N^2>n) = \sum_{k=0}^\infty ((k+1)^2-k^2)P(N>k) = \sum_{k=0}^\infty (2k+1)P(S_{k+1}\le 1). \] We use again the density of the $Gamma(n,1/\lambda)$ distribution, and Fubini to take the integral outside of the sum. We have to compute for $x\in[0,1]$ the sum \[ \sum_{n=0}^\infty \frac{2k+1}{\Gamma(k+1)} \lambda^{k+1} x^k e^{-x\lambda} = 2 \lambda^2 x \exp(x\lambda) e^{-x\lambda} + \lambda \exp(x\lambda) e^{-x\lambda} = (2\lambda^{2}x + \lambda). \] We now integrate the above as in $\int_0^1 (...) dx$; we obtain $E[N^2] = \lambda^{2} + \lambda$.

Finally, $Var[N] = E[N^2]-E[N]^2 = \lambda + \lambda^{2} - \lambda^{2} =\lambda$.

Prove that $N$ has Poisson distribution directly

The above method can also show that $N$ has the Poisson distribution. Since $P(N=n) = P(N >n-1) - P(N>n) = P(S_n \le 1 ) - P(S_{n+1} \le 1)$, we have \begin{align} P(N=n) &= \int_0^1 ( \frac{\lambda^n x^{n-1} e^{-\lambda x}}{\Gamma(n)} - \frac{\lambda^{n+1}x^n e^{-x\lambda}}{\Gamma(n+1)} ) dx \\\ &= \frac{1}{(n-1)!} \int_0^1 e^{-x\lambda} (\lambda x)^{n-1}(1-x\lambda/n) (\lambda dx) \\\ & = \frac{1}{(n-1)!} \int_0^\lambda e^{-u} u^{n-1}(1-u/n) (du). \end{align}

A primitive for the rightmost integrand is $e^{-u}u^n/n$, hence $P(N=n) = \lambda^n e^{-\lambda}/n !$ and $N$ has the Poisson distribution.