9
$\begingroup$

I'm preparing for an interview which requires a decent knowledge of basic probability (at least to get through the interview itself). I'm working through the sheet below from my student days as revision. It's mostly been fairly straightforward, but I am completely stumped on question 12.

http://www.trin.cam.ac.uk/dpk10/IA/exsheet2.pdf

Any help would be appreciated.

Edit: the question is:

Suppose that $X_1, X_2, ... $ are independent identically distributed positive random variables with $\mathbb{E}(X_1) = \mu < \infty$ and $\mathbb{E}(X_1^{-1}) < \infty$. Let $S_n = \sum_{i=1}^n X_i$. Show that $\mathbb{E}(S_m/S_n) = m/n$ when $m<=n$, and $\mathbb{E}(S_m/S_n) = 1 + (m-n)\mu\mathbb{E}(S_n^{-1}))$ when $m>=n$.

In fact, in the process of typing this up, I've solved the second part.

For $m>=n$, $\mathbb{E}(S_m/S_n) = \mathbb{E}(X_1+ . . . +X_m)/\mathbb{E}(X_1+ . . . +X_n)$

$=\mathbb{E}(1 + (X_{n+1} + ... + X_m)/(X_1 + ... + X_n)) $

and the numerator and denominator of the ratio above are clearly independent, so:

$ = 1 + \mathbb{E}(X_{n+1} + ... + X_m)\mathbb{E}(S_n^{-1})$

and we obtain the desired result.

I'm still stuck on the first part though.

$\endgroup$
  • $\begingroup$ It is important that posts be self-contained. Please edit this to include a readable version of the question. We also ask that you indicate what approaches you have tried and what progress, if any, you have made: otherwise we have no basis to gauge the level at which to write the answers. $\endgroup$ – whuber Oct 30 '13 at 17:11
  • $\begingroup$ Updated as requested. $\endgroup$ – Spy_Lord Oct 30 '13 at 17:40
  • 1
    $\begingroup$ Good job! Here's a suggestion for the first part: when you add $n$ identical copies of $S_m/S_n$ together, it looks like the sum will have a distribution whose expectation is easy to compute using only the i.i.d. assumption. $\endgroup$ – whuber Oct 30 '13 at 17:52
  • 1
    $\begingroup$ I appreciate your offer to write it up; I think that would be a useful addition to our site. $\endgroup$ – whuber Oct 30 '13 at 18:31
  • 1
    $\begingroup$ OK I think the step which I thought was right initially, then decided was wrong, is actually OK! Essentially, when you get to the point where you have $\mathbb{E}((nX_1)/(X_1+. . . + X_n))$ then this, by the iid property, is identical to $\mathbb{E}((X_1 + ... + X_n)/(X_1+. . . + X_n)) = 1$ Can you confirm that's OK? If so I'll type it up post-haste. $\endgroup$ – Spy_Lord Oct 30 '13 at 21:08
8
$\begingroup$

Spotting to add $n$ identical copies of $S_m/S_n$ is very clever! But some of us are not so clever, so it is nice to be able to "postpone" the Big Idea to a stage where it is more obvious what to do. Without knowing where to start, there seem be a number of clues that symmetry could be really important (addition is symmetric and we have some summations, and iid variables have the same expectation so maybe they can be swapped around or renamed in useful ways). In fact the "hard" bit of this question is how to deal with the division, the operation which isn't symmetric. How can we exploit the symmetry of summation? From linearity of expectation we have:

$\mathbb{E}(S_m/S_n) = \mathbb{E}\left(\frac{X_1 + ... + X_m}{X_1 + ... + X_n}\right) = \mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right) + ... + \mathbb{E}\left(\frac{X_m}{X_1 + .... + X_n}\right)$

But then on symmetry grounds, given that $X_i$ are iid and $m \le n$, all the terms on the right-hand side are the same! Why? Switch the labels of $X_i$ and $X_j$ for $i, j \le n$. Two terms in the denominator switch position but after reordering it still sums to $S_n$, whereas the numerator changes from $X_i$ to $X_j$. So $\mathbb{E}(X_i/S_n) = \mathbb{E}(X_j/S_n)$. Let's write $\mathbb{E}(X_i/S_n)=k$ for $1 \le i \le n$ and since there are $m$ such terms we have $\mathbb{E}(S_m/S_n) = mk$.

It looks as if $k=1/n$ which would produce the correct result. But how to prove it? We know

$k=\mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right)=\mathbb{E}\left(\frac{X_2}{X_1 + .... + X_n}\right)=...=\mathbb{E}\left(\frac{X_n}{X_1 + .... + X_n}\right)$

It's only at this stage it dawned on me I should be adding these together, to obtain

$nk = \mathbb{E}\left(\frac{X_1}{X_1 + .... + X_n}\right) + \mathbb{E}\left(\frac{X_2}{X_1 + .... + X_n}\right) + ... + \mathbb{E}\left(\frac{X_n}{X_1 + .... + X_n}\right)$ $\implies nk = \mathbb{E}\left(\frac{X_1 + ... + X_n}{X_1 + .... + X_n}\right) = \mathbb{E}(1) = 1$

What's nice about this method is that it preserves the unity of the two parts of the question. The reason symmetry is broken, requiring adjustment when $m>n$, is that the terms on the right-hand side after applying linearity of expectation will be of two types, depending on whether the $X_i$ in the numerator lies in the sum in the denominator. (As before, I can switch the labels of $X_i$ and $X_j$ if both appear in the denominator as this just reorders the sum $S_n$, or if neither does as this clearly leaves the sum unchanged, but if one does and one doesn't then one of the terms in the denominator changes and it no longer sums to $S_n$.) For $i \le n$ we have $\mathbb{E}\left(\frac{X_i}{X_1 + .... + X_n}\right)=k$ and for $i>n$ we have $\mathbb{E}\left(\frac{X_i}{X_1 + .... + X_n}\right)=r$, say. Since we have $n$ of the former terms, and $m-n$ of the latter,

$\mathbb{E}(S_m/S_n) = nk + (m-n)r = 1 + (m-n)r$

Then finding $r$ is straightforward using independence of $S_n^{-1}$ and $X_i$ for $i>n$: $r=\mathbb{E}(X_i S_n^{-1})=\mathbb{E}(X_i) \mathbb{E}(S_n^{-1})=\mu \mathbb{E}(S_n^{-1})$

So the same "trick" works for both parts, it just involves dealing with two cases if $m>n$. I suspect this is why the two parts of the question were given in this order.

$\endgroup$
  • 2
    $\begingroup$ A very nice exposition on your thoughts working through the question, and you make the nk step explicit (my answer sorta just says 'clearly equal'). Cheers! $\endgroup$ – Spy_Lord Oct 31 '13 at 11:25
1
$\begingroup$

Thanks to whuber for the hint for the first part.

Consider $nS_m/S_n$ for the case $m<=n$

We have $\mathbb{E}(nS_m/S_n) = \mathbb{E}((nX_1 + . . . + nX_m)/(X_1 + . . . + X_n))$

$= \mathbb{E}(nX_1/X_1 + . . . + X_n) + . . . + \mathbb{E}(nX_m/X_1 + . . . + X_n)$

and by the iid property, this is equal to:

$m\mathbb{E}((X_1+ . .+ X_n)/(X_1+ . . . + X_n)) = m$

Therefore $\mathbb{E}(S_m/S_n) = m/n$ for $m<=n$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.