Expectation of Quotient of Sums of IID Random Variables (Cambridge University worksheet)

Question

I'm preparing for an interview which requires a decent knowledge of basic probability (at least to get through the interview itself). I'm working through the sheet below from my student days as revision. It's mostly been fairly straightforward, but I am completely stumped on question 12.

http://www.trin.cam.ac.uk/dpk10/IA/exsheet2.pdf

Any help would be appreciated.

Edit: the question is:

Suppose that $X_1, X_2, ... $ are independent identically distributed positive random variables with $\mathbb{E}(X_1) = \mu < \infty$ and $\mathbb{E}(X_1^{-1}) < \infty$. Let $S_n = \sum_{i=1}^n X_i$. Show that $\mathbb{E}(S_m/S_n) = m/n$ when $m<=n$, and $\mathbb{E}(S_m/S_n) = 1 + (m-n)\mu\mathbb{E}(S_n^{-1}))$ when $m>=n$.

In fact, in the process of typing this up, I've solved the second part.

For $m>=n$, $\mathbb{E}(S_m/S_n) = \mathbb{E}(X_1+ . . . +X_m)/\mathbb{E}(X_1+ . . . +X_n)$

$=\mathbb{E}(1 + (X_{n+1} + ... + X_m)/(X_1 + ... + X_n)) $

and the numerator and denominator of the ratio above are clearly independent, so:

$ = 1 + \mathbb{E}(X_{n+1} + ... + X_m)\mathbb{E}(S_n^{-1})$

and we obtain the desired result.

I'm still stuck on the first part though.

Answer 1

Spotting to add $n$ identical copies of $S_m/S_n$ is very clever! But some of us are not so clever, so it is nice to be able to "postpone" the Big Idea to a stage where it is more obvious what to do. Without knowing where to start, there seem be a number of clues that symmetry could be really important (addition is symmetric and we have some summations, and iid variables have the same expectation so maybe they can be swapped around or renamed in useful ways). In fact the "hard" bit of this question is how to deal with the division, the operation which isn't symmetric. How can we exploit the symmetry of summation? From linearity of expectation we have:

$\mathbb{E}(S_m/S_n) = \mathbb{E}\left(\frac{X_1 + ... + X_m}{X_1 + ... + X_n}\right) = \mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right) + ... + \mathbb{E}\left(\frac{X_m}{X_1 + .... + X_n}\right)$

But then on symmetry grounds, given that $X_i$ are iid and $m \le n$, all the terms on the right-hand side are the same! Why? Switch the labels of $X_i$ and $X_j$ for $i, j \le n$. Two terms in the denominator switch position but after reordering it still sums to $S_n$, whereas the numerator changes from $X_i$ to $X_j$. So $\mathbb{E}(X_i/S_n) = \mathbb{E}(X_j/S_n)$. Let's write $\mathbb{E}(X_i/S_n)=k$ for $1 \le i \le n$ and since there are $m$ such terms we have $\mathbb{E}(S_m/S_n) = mk$.

It looks as if $k=1/n$ which would produce the correct result. But how to prove it? We know

$k=\mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right)=\mathbb{E}\left(\frac{X_2}{X_1 + .... + X_n}\right)=...=\mathbb{E}\left(\frac{X_n}{X_1 + .... + X_n}\right)$

It's only at this stage it dawned on me I should be adding these together, to obtain

$nk = \mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right) + \mathbb{E}\left(\frac{X_2}{X_1 + .... + X_n}\right) + ... + \mathbb{E}\left(\frac{X_n}{X_1 + .... + X_n}\right)$ $\implies nk = \mathbb{E}\left(\frac{X_1 + ... + X_n}{X_1 + .... + X_n}\right) = \mathbb{E}(1) = 1$

What's nice about this method is that it preserves the unity of the two parts of the question. The reason symmetry is broken, requiring adjustment when $m>n$, is that the terms on the right-hand side after applying linearity of expectation will be of two types, depending on whether the $X_i$ in the numerator lies in the sum in the denominator. (As before, I can switch the labels of $X_i$ and $X_j$ if both appear in the denominator as this just reorders the sum $S_n$, or if neither does as this clearly leaves the sum unchanged, but if one does and one doesn't then one of the terms in the denominator changes and it no longer sums to $S_n$.) For $i \le n$ we have $\mathbb{E}\left(\frac{X_i}{X_1 + .... + X_n}\right)=k$ and for $i>n$ we have $\mathbb{E}\left(\frac{X_i}{X_1 + .... + X_n}\right)=r$, say. Since we have $n$ of the former terms, and $m-n$ of the latter,

$\mathbb{E}(S_m/S_n) = nk + (m-n)r = 1 + (m-n)r$

Then finding $r$ is straightforward using independence of $S_n^{-1}$ and $X_i$ for $i>n$: $r=\mathbb{E}(X_i S_n^{-1})=\mathbb{E}(X_i) \mathbb{E}(S_n^{-1})=\mu \mathbb{E}(S_n^{-1})$

So the same "trick" works for both parts, it just involves dealing with two cases if $m>n$. I suspect this is why the two parts of the question were given in this order.

Answer 2

Thanks to whuber for the hint for the first part.

Consider $nS_m/S_n$ for the case $m<=n$

We have $\mathbb{E}(nS_m/S_n) = \mathbb{E}((nX_1 + . . . + nX_m)/(X_1 + . . . + X_n))$

$= \mathbb{E}(nX_1/X_1 + . . . + X_n) + . . . + \mathbb{E}(nX_m/X_1 + . . . + X_n)$

and by the iid property, this is equal to:

$m\mathbb{E}((X_1+ . .+ X_n)/(X_1+ . . . + X_n)) = m$

Therefore $\mathbb{E}(S_m/S_n) = m/n$ for $m<=n$