How to 'read' (understand ) an expected value equation (example inside)

Question

I have just come across expected values and they are giving me a bit of grief trying to understand them.

e.g. for covariance the equation is $\text{E}\left((x - \bar{x})(y - \bar{y})\right)$

To me this means the expected value is take my x value , subtract the mean, and then multiply it by the y value minus the y mean.

What I don't get is what in this equation says to do this for every pair of xy values i have? I cant see a symbol in it that says to sum everything up ... $\Sigma$. How do we know to sum everything up ?

Also looking at the using a 4 sided dice, lets look at the expected value of our roll.

E = 1/4 * 1 + 1/4 * 2 + 1/4 * 3 + 1/4 * 4 = 2.5

Here I have multiplied each possible value by its probability of occurring. This is what I understood an Expected value to be, but I don't get how this relates to $\text{E}((x - \bar x)(y -\bar y))$ ...??

Answer 1

To me this means the expected value is take my x value , subtract the mean, and then multiply it by the y value minus the y mean.

No, you're dealing with covariance here. So the covariance is "take my x value , subtract the mean, and then multiply it by the y value minus the y mean"... then take the expectation of that.

What I don't get is what in this equation says to do this for every pair of xy values I have?

You must take great care not to conflate sample values and results for random variables themselves (notionally population quantities, in effect).

You wrote an expression for the covariance of univariate random variables $X$ and $Y$.

Roughly speaking, when dealing with the sample quantities, replace expectations with sample averages, which are scaled sums.

Also looking at the using a 4 sided dice, lets look at the expected value of our roll.

Here I have multiplied each possible value by its probability of occurring. This is what I understood an Expected value to be, but I don't get how this relates to $\text{E}((x - \bar x)(y -\bar y))$ ...??

Here you're confusing expectation with covariance. If you want a covariance, what two things do you want the covariance of?

Answer 2

(Note: I'm assuming we are dealing with discrete random variables, for continuous random variables the expected value is not obtained by summing products of probabilities and outcomes).

You give the correct expression for computing the expected value of a 4-sided die (assuming all sides have equal probability). That is, you get the expected value of a roll of your 4-sided dice by computing the sum of all possible values of the roll multiplied by their respective probabilities. Similarly,

1. The expected value of $x$, $E(x)$ is obtained by summing all possible values of $x$ multiplied by their probabilities.
2. The expected value of garble, $E(\textrm{garble})$, is obtained by summing all possible values of garble multiplied by their probabilities.
3. The expected value of fnorble, $E(\textrm{fnorble})$, is obtained by summing all possible values of fnorble multiplied by their probabilities.
4. The expected value of $(x-\bar{x})(y-\bar{y})$, $E((x-\bar{x})(y-\bar{y})),$ is obtained by summing all possible values of $(x-\bar{x})(y-\bar{y})$ multiplied by their probabilities.

That is, to understand the definition of covariance, see that we are taking the expectation of some random variable, namely $(x-\bar{x})(y-\bar{y})$ and then apply the definition of expectation.

Let us work out an example. Let $x$ be the 4-sided die considered in the question. Let $y$ be determined by tossing a fair coin (heads $=1$, tails $=0$). (Assume the coin toss and the die roll are independent, which seems a natural assumption). We now compute $Cov(x,y)$, that is, the expected value of $(x-\bar{x})(y-\bar{y})$. To do this, we need to consider all possible values of $(x-\bar{x})(y-\bar{y})$. But this depends on both $x$ and $y$. Therefore, to find out the possible values of $(x-\bar{x})(y-\bar{y})$, we must look at all possible combinations of $x$ and $y$. This is done in the following table. Note that $\bar{x}=2.5$ was computed in the question and $\bar{y}=0.5\times0 + 0.5\times1 = 0.5$

\begin{equation} \begin{array}{c|c|c|c|c} x & y & (x-\bar{x}) & (y-\bar{y}) & (x-\bar{x})(y-\bar{y}) \\ \hline 1 & 0 & -1.5 & -0.5 & 0.75 \\ 2 & 0 & -0.5 & -0.5 & 0.25 \\ 3 & 0 & 0.5 & -0.5 & -0.25 \\ 4 & 0 & 1.5 & -0.5 & -0.75 \\ 1 & 1 & -1.5 & 0.5 & -0.75 \\ 2 & 1 & -0.5 & 0.5 & -0.25 \\ 3 & 1 & 0.5 & 0.5 & 0.25 \\ 4 & 1 & 1.5 & 0.5 & 0.75 \end{array} \end{equation} So, there are 4 different values $(x-\bar{x})(y-\bar{y})$ can take: $(-0.75,-0.25,0.25,0.75)$. Each of these may result from two different $(\textrm{die roll, coin toss})=(x,y)$ combinations, thus the probability of each value occurring is $2\times \frac{1}{4}\times\frac{1}{2} = \frac{1}{4}$. Now we are ready to compute the expected value of $(x-\bar{x})(y-\bar{y})$: \begin{equation} E((x-\bar{x})(y-\bar{y})) = \frac{1}{4}\left(-0.75\right) + \frac{1}{4}\left(-0.25\right) + \frac{1}{4}0.25 + \frac{1}{4}0.75 = 0. \end{equation} That is, the covariance of $x$ and $y$ is $0$. But this was expected, as the coin toss and die roll are independnet, and the covariance of two independent random variables is known to be always 0.