6
$\begingroup$

Given a multivariate random variable $\mathbf{X}=(X_1, ..., X_n)^\intercal : \Omega \rightarrow \mathbb{R}^n$ I want to determine the expectation value of this RV. Now wikipedia says the expectation is simply given by: $$\mathbb{E}[\mathbf{X}]=(\mathbb{E}[X_1], ..., \mathbb{E}[X_n])^\intercal.$$ However, the distribution of a random vector is the joint distribution of all its components, i.e., the distribution of $\mathbf{X}$ is $\mathbb{P}_{X_1, ..., X_n}$. Hence, for instance for $n=2$ if $X_1$ and $X_2$ are discrete the expected value would be computed like: \begin{align} \mathbb{E}[\mathbf{X}]&= \sum_{i=1}^\infty \mathbf{X}_i \mathbb{P}_{X_1,X_2}(x_1^{(i)}, x_2^{(i)})\\ &=\sum_{i=1}^\infty \binom{x_1^{(i)}}{x_2^{(i)}} \mathbb{P}_{X_1,X_2}(x_1^{(i)}, x_2^{(i)})\\ &=\binom{\sum_{i=1}^\infty x_1^{(i)} \mathbb{P}_{X_1,X_2}(x_1^{(i)}, x_2^{(i)})}{\sum_{i=1}^\infty x_2^{(i)} \mathbb{P}_{X_1,X_2}(x_1^{(i)}, x_2^{(i)})} \end{align} And this doesnt occur to me to be the same as $$\mathbb{E}[\mathbf{X}]=(\mathbb{E}[X_1], \mathbb{E}[X_2])^\intercal.$$ How do these two definitions match?

$\endgroup$
6
  • 3
    $\begingroup$ Keep going with your derivation: write those sums in terms of the marginal distributions of $\mathbf X.$ $\endgroup$
    – whuber
    Commented Mar 4, 2022 at 14:16
  • $\begingroup$ So I wanted to do that but for marginalization I would require another sum that does not depend on the respective variable right? I.e., like this I cannot marginalize because I also always have to sum over the sample point $x_k^{(i)}$, k=1,2. $\endgroup$
    – guest1
    Commented Mar 4, 2022 at 14:19
  • 2
    $\begingroup$ I see no obstacle to marginalizing. I would invite you to tackle a tiny concrete case. How about making a table of the values and probabilities of such a distribution when both marginal variables are binary? Working through this (which should take only a minute or two) will reveal what happens generally. $\endgroup$
    – whuber
    Commented Mar 4, 2022 at 14:34
  • $\begingroup$ hmm but if they are not independent then I also have to define the conditional probabilites to get the joint distribution, no? $\endgroup$
    – guest1
    Commented Mar 4, 2022 at 15:20
  • $\begingroup$ But by properly conditioning you can eliminate the nuisance component of the joint in the sum... $\endgroup$
    – Xi'an
    Commented Mar 4, 2022 at 15:43

1 Answer 1

9
+50
$\begingroup$

It looks like you might be getting tripped up with the indexing and summations. Here's how to handle the 2d discrete example using the same approach you were trying to take. The same logic applies to higher dimensions and/or continuous variables (in that case, replace sums with integrals).

Notation

Let $x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$ be a realization of the random vector $X = \begin{bmatrix} X_1 \\ X_2 \end{bmatrix}$. I'll use the shorthand $p(x_1, x_2)$ for the joint probability $Pr(X_1=x_1, X_2=x_2)$, and $p(x_1)$ for the marginal probabillity $Pr(X_1=x_1)$. Sums will be taken over all possible values of each variable. For example, if $X_1$ has range $\mathcal{X}_1$ then $\sum_{x_1}$ is shorthand for $\sum_{x_1 \in \mathcal{X}_1}$.

Expectation

$$E[X] = \sum_x x p(x)$$

$$= \sum_{x_1} \sum_{x_2} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} p(x_1, x_2)$$

$$= \sum_{x_1} \sum_{x_2} \begin{bmatrix} x_1 p(x_1, x_2) \\ x_2 p(x_1, x_2) \\ \end{bmatrix}$$

$$= \begin{bmatrix} \sum_{x_1} \sum_{x_2} x_1 p(x_1, x_2) \\ \sum_{x_1} \sum_{x_2} x_2 p(x_1, x_2) \\ \end{bmatrix}$$

$$= \begin{bmatrix} \sum_{x_1} \sum_{x_2} x_1 p(x_1, x_2) \\ \sum_{x_2} \sum_{x_1} x_2 p(x_1, x_2) \\ \end{bmatrix}$$

$$= \begin{bmatrix} \sum_{x_1} x_1 \sum_{x_2} p(x_1, x_2) \\ \sum_{x_2} x_2 \sum_{x_1} p(x_1, x_2) \\ \end{bmatrix}$$

$$= \begin{bmatrix} \sum_{x_1} x_1 p(x_1) \\ \sum_{x_2} x_2 p(x_2) \\ \end{bmatrix}$$

$$= \begin{bmatrix} E[X_1] \\ E[X_2] \\ \end{bmatrix}$$

$\endgroup$
3
  • $\begingroup$ Thanks for the answer, but why can you rewrite the sum over $x$ as two sums over $x_1$ and $x_2$. I mean there should be the same amount of $x_1$ as there are $x_2$ and this should be the same as the amount of vectors $x$ since the $x_1$ and $x_2$ are entries of the vector $x$, no? $\endgroup$
    – guest1
    Commented Mar 7, 2022 at 16:47
  • 2
    $\begingroup$ @guest1 Note that the sums over $x_1$ and $x_2$ are nested, not separate. For example, in the binary case, we need to take a sum $\sum_{x \in \mathcal{X}} \cdots$ over all possible vectors $\mathcal{X} = \big\{ [0,0], [0,1], [1,0], [1,1] \big\}$. This can be written as a nested sum over the individual components: $\sum_{x_1 \in \{0,1\}} \sum_{x_2 \in \{0,1\}} \cdots$. We have two possible values for $x_1$ and two possible values for $x_2$, so the nested sum has 4 terms--one for each possible vector. $\endgroup$
    – user20160
    Commented Mar 7, 2022 at 18:13
  • 1
    $\begingroup$ @guest1 To be more specific, a nested sum over binary values would be written out as: $\sum_{x_1 \in \{0,1\}} \sum_{x_2 \in \{0,1\}} f(x_1, x_2) = f(0,0) + f(0,1) + f(1,0) + f(1,1)$ $\endgroup$
    – user20160
    Commented Mar 7, 2022 at 18:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.