Due to the linearity property of the expectations, we can write $E(X+Y) = E(X)+E(Y)$. But in Chapter 6.1 of the book, "Counterexamples in probability", the writer, Jordan Stoyanov argued that this is not true for three random variables, i.e., for $E(X+Y+Z)$.
The paper cited as the reference in that chapter is "An Unexpected Expectation"(1977) by Gordon Simons. But unfortunately, in that paper, the writer didn't provide any formal proof for this phenomenon. And stated that "We have no explanations why the theorem given above should hold for two variables but fail for three."
I haven't quite understood why this is the case or found any proof for this. Can anyone please help me understand why the linearity property will not be held for the case of three random variables? And please correct me if I am getting it totally wrong.
-
2$\begingroup$ Re "not found any proof:" the introductory paragraph is clear. By exhibiting such sets of variables $(X,Y,Z)$ this paper offers to demonstrate by counterexample that the hypothesis ("...depend only ... marginal distributions") cannot be true. That is as formal as one can get! Also, you misinterpret this paper: it does not argue that expectation fails to be linear (because such an argument would be erroneous). $\endgroup$– whuber ♦Commented Mar 31, 2021 at 16:56
1 Answer
Simons' counterexample considers random variables that don't have finite marginal expectations. If you sum 3 (or more) random variables that have finite marginal expectations, then the expectation of their sum equals the sum of their expectations. So to say "this is not true for three random variables" is not quite right.
Based on a comment from the OP, the content below is edited from my original response.
What I interpret Simons to be claiming is that if $E[X+Y]^+$ or $E[X+Y]^-$ is finite, then to calculate $E[X+Y]$ you do not need to know or assume anything about how $X$ and $Y$ are jointly distributed. Rather, you only need to know about marginal distributions. But, he says, this doesn't extend to the case of three or more variables. In other words, the finitude of $E[X+Y+Z]^+$ or $E[X+Y+Z]^-$ does not imply that one can calculate $E[X+Y+Z]$ using only knowledge of the marginal distributions of $X$, $Y$, and $Z$. He gives a counterexample to demonstrate this point.
The result only seems to contradict the usual thing we teach ("expectations are additive") because we are implicitly assuming that the marginal expectations are finite. Simons is allowing for non-finite marginal expectations.
When Simon writes We have no explanations why the theorem given above should hold for two variables but fail for three, I understand this to be him saying that he does have a good intuitive explanation for why this mathematical result - which he has shown rigorously - holds.
-
$\begingroup$ Thank you @psboonstra. I have noticed that the problem only arises when they do not have finite marginal. I was wondering how this case is tackled when there are two variables? $\endgroup$ Commented Mar 31, 2021 at 19:01
-
1$\begingroup$ For two variables, he gives a direct proof. By creating truncated versions $X^c$ and $Y^c$ that have finite expectation, he can fall back on the usual intuition. The limit at the top of page 158, combined with the fact that that $E[X^c+Y^c]=E[X^c]+E[Y^c]$, gives the result. $\endgroup$ Commented Mar 31, 2021 at 20:08
-
$\begingroup$ sorry I am new to this -- can you explain what $E[X+Y]^{+}$, $E[X+Y]^{-}$ and $X^c$ mean in your notation? $\endgroup$ Commented Apr 15, 2021 at 15:38
-
$\begingroup$ $E[X+Y]^+$ and $E[X+Y]^-$ are the expectations of the positive and negative parts of $X+Y$, respectively. en.wikipedia.org/wiki/Positive_and_negative_parts. $X^c$ is a truncated version of $X$ that Simons creates which is equal to $X$ when $|X| \leq c$, equal to $-c$ when $X<(-c)$, and equal to $c$ when $X>c$. $\endgroup$ Commented Aug 8, 2021 at 18:05