Understand the linearity of the expected value operator

Question

Let Z = X + Y. The linearity of $\mathsf E$ implies that $\mathsf E [Z] = \mathsf E [X] + \mathsf E [Y]$. The left-hand side should be $\int (x+y)p_{x+y}(x+y)d(x+y)$. The right-hand side should be $\int x p_{x}(x) dx + \int y p_{y}(y) dy$.

Then how to make left-hand side equals to right-hand side? (Prove these two integrals are equal)

(I know this question is stupid. If I write left-hand side like $\int (x+y) p(x,y)dxdy$ , this problem may be solved already. But I still want to ask, if I just rewrite the left-hand side formula like the above one, is this rewriting wrong? Or can I prove these two integral equal?)

Update 1:

I see one comment from https://math.stackexchange.com/questions/1810365/proof-of-linearity-for-expectation-given-random-variables-are-dependent

\begin{align}\mathsf E(X+Y)&=\int_\Bbb R z~f_{X+Y}(z)\mathsf d z\\&=\iint_{\Bbb R^2} z~f_{X+Y,Y}(z,y)\mathsf d z\mathsf d y\\&= \iint_{\Bbb R^2} z~f_{X,Y}(z-y,y)\mathsf d z\mathsf d y\\&=\iint_{\Bbb R^2} (x+y)~f_{X,Y}(x,y)\mathsf d x\mathsf d y\\&\vdots\\&=\mathsf E(X)+\mathsf E(Y)\end{align}

Can someone explain this? Especially second step suddenly introduce one more variable becoming 2 dimensional integral and the third step replace $~f_{X+Y,Y}(z,y)$ to $~f_{X,Y}(z-y,y)$. I am quite confused of the change of dz -> dzdy -> dxdy, which seems like change freely...

Because in my point of view I may write \begin{align}\mathsf E(X+Y)&=\int_\Bbb R (x+y)~f_{X+Y}(x+y)\mathsf d(x+y)\\&= \int_\Bbb R (x+y)~f_{X+Y}(x+y)\mathsf (dx+dy) \\&= \int_\Bbb R ~f_{X+Y}(x+y) (xdx+ydx+xdy+ydy)\\&\vdots \end{align}

I believe this one must be wrong, but I don't know which part of these steps is wrong...

Update 2:

Thank you all for your explanation about why I should not write $d(x+y) = dx + dy$. I understand that $dx$ should be interpreted jointly with $\int$ rather than separately.

But in practice, I remember I saw some problem is solved by playing around $\int f(x) d g(x)$ by rewrite the $dg(x)$ part. (I am not sure, but I saw something like $\int f(x) d g(x) = \int f(x) g^{\prime}(x) dx$ is done.)

I guess this is a bit out of the scope for the original question, but if it is possible, can anyone explain to me why this kind of rewriting happens? And is there any rule for this kind of rewriting?

Thank you all again for your patience.

Answer 1

If one defines $Z=X+Y$, the convolution formula expresses the density of $Z$ as $$p_Z(z)=\int p_{X,Y}(x,z-x)\,\text dx$$ Hence $$\mathbb E[Z]=\int zp_{Z}(z)\text dz=\iint zp_{X,Y}(x,z-x)\,\text dx\,\text dz=\iint (x+y)p_{X,Y}(x,y)\,\text dx\,\text dy$$ by a change of variable in the final equality. Then, by linearity of the integral $$\mathbb E[Z]=\iint xp_{X,Y}(x,y)\,\text dx\,\text dy+\iint yp_{X,Y}(x,y)\,\text dx\,\text dy\quad\\ \quad=\iint xp_{X}(x)p_{Y|X}(y|x)\,\text dx\,\text dy+\iint yp_{Y}(y)p_{X|Y}(x|y)\,\text dx\,\text dy\\ =\int xp_{X}(x)\,\text dx+\int yp_{Y}(y)\,\text dy\qquad\qquad\qquad\qquad $$

Answer 2

A minimalist proof of the linearity of the expected value operator

Indeed, there are a lot of inexplicit steps in this calculation! Let's define the random variable $Z = X + Y$ and derive $\mathsf E[Z]$:

\begin{align} &\mathsf E[Z]\\ &=\int_\Bbb R z~p(z) ~ \mathsf d z && \text{by definition of the expected value}\\ &=\int_\Bbb R z ~ \Big( \int_\Bbb R p(z \cap y) ~ dy \Big) \mathsf d z && \text{by the law of total probability}\\ &=\int_\Bbb R z ~ \Big( \int_\Bbb R p(z | y) ~ p(y) ~ dy \Big) \mathsf d z && \text{by definition of the conditional probability}\\ &=\int_\Bbb R z ~ \Big( \int_\Bbb R p(z - y | y) ~ p(y) ~ dy \Big) \mathsf d z && \text{since p(Z=x+y|Y=y) = p(X=x|Y=y)}\\ &=\int_\Bbb R \Big( \int_\Bbb R z ~ p(z - y | y) ~ p(y) ~ dy \Big) \mathsf d z && \text{by linearity of the integral}\\ &=\int_\Bbb R \Big( \int_\Bbb R (x + y) ~ p(x | y) ~ p(y) ~ dy \Big) \mathsf d x && \text{by substituting $x = z - y$ (note that $dx = dz$)} \end{align}

By linearity of the integral, this last line is equal to $$\int_\Bbb R x ~ \Big( \int_\Bbb R p(x | y) ~ p(y) ~ dy \Big) \mathsf d x + \int_\Bbb R \Big( \int_\Bbb R y ~ p(x | y) ~ p(y) ~ dy \Big) \mathsf d x$$

Let's deal with the first term: \begin{align} \int_\Bbb R x ~ \Big( \int_\Bbb R p(x | y) p(y) dy \Big) \mathsf d x &=\int_\Bbb R x \Big( \int_\Bbb R p(x \cap y) dy \Big) \mathsf d x && \text{conditional prob}\\ &=\int_\Bbb R x p(x) \mathsf d x && \text{total prob}\\ &=\mathsf E[X] \end{align}

Let's deal with the second term. We'll make the reasonable assumption that X and Y are measurable so as to use Fubini's theorem. This theorem allows us to change the order of integrals: \begin{align} \int_\Bbb R \Big( \int_\Bbb R y ~ p(x | y) ~ p(y) ~ dy \Big) \mathsf d x &=\int_\Bbb R y \Big( \int_\Bbb R p(y) ~ p(x | y) ~ dx \Big) \mathsf d y &&\text{with Fubini's theorem}\\ &=\int_\Bbb R y \Big( \int_\Bbb R p(x \cap y) ~ dx \Big) \mathsf d y &&\text{conditional prob}\\ &=\int_\Bbb R y ~ p(y) ~ \mathsf d y &&\text{total prob}\\ &=\mathsf E[Y] \end{align}

About $dz$

Because in my point of view I may write \begin{align}\mathsf E(X+Y)&=\int_\Bbb R (x+y)~f_{X+Y}(x+y)\mathsf d(x+y)\\&= \int_\Bbb R (x+y)~f_{X+Y}(x+y)\mathsf (dx+dy) \\&= \int_\Bbb R ~f_{X+Y}(x+y) (xdx+ydx+xdy+ydy)\\&\vdots \end{align}

Can someone explain this? Especially second step suddenly introduce one more variable becoming 2 dimensional integral and the third step replace 𝑓𝑋+𝑌,𝑌(𝑧,𝑦) to 𝑓𝑋,𝑌(𝑧−𝑦,𝑦) . I am quite confused of the change of dz -> dzdy -> dxdy, which seems like change freely...

Indeed, as you correctly stated, one cannot change the order of these notations freely, nor can they separate $d(x+y)$ into $dx + dy$.

In my opinion, you can think of $dz$ as the part of these notations that indicates what is the name of the variable that "scans" the space over which you are integrating. Since you are integrating once over $\mathbb{R}$, $d(x+y)$ is a confusing notation which gives you the impression that you could split it in two ($dx + dy$), when in fact it is not the case.

Besides, when substituting a variable for another, it's in fact a function of the variable that you are substituting. Check the wikipedia article about substitution for what I think is a clear formulation: https://en.wikipedia.org/wiki/Integration_by_substitution#Statement_for_definite_integrals

In our case, we are substituting $x$ to $\phi(z) = z - y$, and $dx$ is equal to $\frac{d\phi}{dz}(z) ~ dz = 1 \cdot dz$.