Finding $\mathrm{Var}(N)$ if $N=\inf\{n\ge1:\sum_{i=1}^nX_i>1\}$ where $X_i$'s are i.i.d Exponential variables Suppose $X_1, X_2, X_3, \ldots, X_n$ be independent and identically distributed random variables having an exponential distribution with mean $\frac{1}{\lambda}$.
If $S_n = X_1 + X_2 + \ldots + X_n$ and $N = \inf\{n \geq 1: S_n > 1\}$, then what would be the value of $\mathrm{Var}(N)$ ?
 A: I am going to use the definition $N=\inf \{ n\ge0: S_{n+1}> 1 \}$ which seems
more natural.
The event $\{N\le n\}$ is the same as $\{S_{n+1} > 1\}$ (or equivalently, $\{N>n\}$ is the same as $\{S_{n+1} \le 1\}$).
By the tail-sum formula, we can thus get the expectation of $N$ as
\[
E[N] = \sum_{n=0}^\infty P(N>n) = \sum_{n=0}^\infty P(S_{n+1}\le 1).
\]
Now for any fixed $n$, $S_n$ has the $Gamma(n,1/\lambda)$ distribution (because it's a sum of exponential random variables) and by looking up the density of such distribution, we get
\[
P(S_{n+1} \le 1) = \int_0^1 \frac{1}{\Gamma(n+1)}\lambda^{n+1}x^{n}e^{-x\lambda}dx = 
 \lambda \int_0^1 \frac{1}{n!}(x\lambda)^{n}e^{-x\lambda}dx
\]
Since $\exp(x\lambda)=\sum_{n=0}^\infty (x\lambda)^{n}/n!$, we obtain by moving the integral outside of sum by Fubini's theorem
\[
E[N] = \lambda \int_0^1 \exp(x\lambda) e^{-x\lambda} dx = \lambda.
\]
We can compute $E[N^2]$ similarly using the tail formula, by making "packets"
\[
E[N^2] = \sum_{n=0}^\infty P(N^2>n) = \sum_{k=0}^\infty ((k+1)^2-k^2)P(N>k)
= \sum_{k=0}^\infty (2k+1)P(S_{k+1}\le 1).
\]
We use again the density of the $Gamma(n,1/\lambda)$ distribution, and Fubini to take the integral outside of the sum. We have to compute for $x\in[0,1]$ the sum
\[
\sum_{n=0}^\infty \frac{2k+1}{\Gamma(k+1)} \lambda^{k+1} x^k e^{-x\lambda} = 2 \lambda^2 x \exp(x\lambda) e^{-x\lambda}
+ \lambda \exp(x\lambda) e^{-x\lambda} = (2\lambda^{2}x + \lambda).
\]
We now integrate the above as in $\int_0^1 (...) dx$; we obtain $E[N^2] = \lambda^{2} + \lambda$.
Finally, $Var[N] = E[N^2]-E[N]^2 = \lambda + \lambda^{2} - \lambda^{2} =\lambda$.
Prove that $N$ has Poisson distribution directly
The above method can also show that $N$ has the Poisson distribution.
Since $P(N=n) = P(N >n-1) - P(N>n) = P(S_n \le 1 ) - P(S_{n+1} \le 1)$, we have
\begin{align}
P(N=n) 
&= \int_0^1 ( \frac{\lambda^n x^{n-1} e^{-\lambda x}}{\Gamma(n)} - \frac{\lambda^{n+1}x^n e^{-x\lambda}}{\Gamma(n+1)} ) dx
\\\ &= \frac{1}{(n-1)!} \int_0^1 e^{-x\lambda} (\lambda x)^{n-1}(1-x\lambda/n) (\lambda dx)
\\\ & = \frac{1}{(n-1)!} \int_0^\lambda e^{-u} u^{n-1}(1-u/n) (du).
\end{align}
A primitive for the rightmost integrand is $e^{-u}u^n/n$, hence
$P(N=n) = \lambda^n e^{-\lambda}/n !$ and $N$ has the Poisson distribution.
