15
$\begingroup$

The expected value of a distribution $f(x)$ is the mean, that is the weighted average value $$E[x]=\int_{-\infty}^{+\infty} x \, \, f(x) dx$$

The most likely value is the mode, that is the most probable value.

However do we expect somehow to see $E[x]$ a lot of times? Quoting from here:

If the outcomes $x_i$ are not equally probable, then the simple average must be replaced with the weighted average, which takes into account the fact that some outcomes are more likely than the others. The intuition however remains the same: the expected value of $x$ is what one expects to happen on average.

I cannot understand what does "happen on average" means, does this mean that, for istance, taking a measure a lot of time I expect to see $E[x]$ more than other values of $x$? But isn't this the definition of mode?

So how to interpret the statement? And what is the probabilistic meaning of $E[x]$?


I would also like to show an example where I do get confused. Studying $\chi^2$ distribution I learned that the mode is $\chi^2_{mode}=\nu-2$, while $E[\chi^2]=\nu$, where $\nu$ are the degrees of freedom of data.

I heard at university that, when doing a $\chi^2$ test after using Least Squares Method to fit a set of data, I should expect to get $\chi^2 \approx \nu$ because "that's what happens in general".


Did I misunderstand all of this or is the expected value somehow very probable? (Even if the most probable value is of course the mode)

$\endgroup$
  • 4
    $\begingroup$ I really like the power of the tickets-in-a-box metaphor for this question, because it produces a simple, clear, answer: the expectation of a random variable is the sum of its values (as drawn on the tickets) divided by the count of the tickets. That's it. Any statement that does not follow from this definition (or more sophisticated mathematical equivalents of it) is just a heuristic and might very well be incorrect in some circumstances. $\endgroup$ – whuber Aug 14 '16 at 18:29
18
$\begingroup$

For a Normal distribution, the expected value, a.k.a. the mean, equals the mode.

In general, not only is the expected value not only not the most likely (or at highest density), but it may have no chance of occurring. For instance, consider the random variable X which equals 0 or 2, each with probability 0.5. Then EX = 1, but the expected value,1, has 0 probability of occurring, while 0 and 2 are both modes of the distribution.

The quote "the expected value of x is what one expects to happen on average" is non-technical layman's language, which as evident by your confusion, only serves to confuse matters. Expected value has a very specific meaning in probability as being the mathematical average. Whereas in layman's language, an expected value or "on average" may be something expected to typically occur. These can be reconciled if "on average" is interpreted as being the mathematical average of what occurs.

Expectantly yours,

Joe Average

$\endgroup$
  • 1
    $\begingroup$ Begs the question: What about the median, which is guaranteed to be possible? $\endgroup$ – bright-star Aug 15 '16 at 3:29
  • $\begingroup$ As @TrevorAlexander said, mode does not give guarantees either. Consider mode of continuous distribution. $\endgroup$ – Tim Aug 15 '16 at 8:52
  • 3
    $\begingroup$ @Trevor Alexander There is always a median which is possible (positive probability or density). However, not all medians are necessarily possible. A median of the random variable X is any point m for which $P(X \le m) \ge 1/2$ and $P(X \ge m) \ge 1/2$. If X equals 1,2,3, or 4, each with probability 1/4, then any number in the interval [2,3] is a median of X. $\endgroup$ – Mark L. Stone Aug 15 '16 at 12:16
5
$\begingroup$

The expected value is a priori very abstract and there is no reason to think that it's the most probable outcome; as other have point out, it's easy to construct random variables for which $$P( X = E(X) ) = 0$$ (and the same with density if $X$ is continuos)

The only justification for the expected value, and the reason why we "expect to see it often", is the Law of large numbers:

if you have $n$ independent identically distributed variables $X_i$, then

$$\frac {X_1 + \dots + X_n}{n} \to E(X)$$

(for a suitable meaning of $\to$ which is pointless to investigate at the moment)

What does it mean? Imagine that you throw a coin with probability $p> \frac 12$ of landing head, which we will associate with the number $1$, and probability $1-p$ of landing tail (that is, $0$). What is the most probable outcome? 1! (that is, head) What is the expected value? $$E(X) = 1\cdot p + 0\cdot(1-p) = p$$

Now clearly "p" will never happen (it's either head or tail, either 0 or 1).

But launch the coin 10.000 times ,and record the times it came up head over the total number of throws. This number captures what we intuitively think of average ("average number of heads"). And the law of large numbers tells you that this number will be close to $E(X) = p$

$\endgroup$
  • $\begingroup$ I wouldn't say the law of large numbers is the only justification for the expected value. For example, en.wikipedia.org/wiki/… is a justification for considering expected values of utility functions (I haven't studied the proof but I'm surprised if it's somehow based on law of large numbers). $\endgroup$ – Juho Kokkala Aug 15 '16 at 13:22
3
$\begingroup$

I don't like the term "expected value" and didn't use it when teaching probability. "Arithmetic mean" is better, in my opinion, because the arithmetic mean of a 6-sided die is 3.5 but such a number doesn't occur. I did originally hear the term "expectation value" for the concept when in college. Lots of technical terms do not agree with the obvious non-technical meaning. ("Or" comes to mind.)

Note that a distribution may have more than one mode but the arithmetic mean is unique. Mode, mean, and median are different and have different uses.

$\endgroup$
  • 1
    $\begingroup$ Nice one on the "or". That made me think of my course in Linear Programming in which we studied several Theorems of the Alternative. They were of the form "Either A is true or B is true, but not both". It's much easier to express it as A xor B. I don't hear much use of xor in casual street conversation. $\endgroup$ – Mark L. Stone Aug 14 '16 at 22:08
2
$\begingroup$

The difference is easiest to see with discrete distributions:

Consider two sets of values where each number is equally likely to be drawn: {1,2,2,2,10} and {1,2,2,2,3}.

Both have the same mode (2), but the expected values differ. Expected value puts extra weight on large values while the mode simply looks for what value occurs frequently. So if you drew from this distribution a bunch of times, your sample average would be close to the expected value, while the most common integer to occur would be the close to the mode.

The mode is defined as $mode=arg{\max{f(x)}}$ while as you showed above, the expected value integrates over $x*f(x)$ so it considers the weight of each x.

The use of language to distinguish between different measure of central tendency is a common issue when learning statistics. For example, the median is another measure that isn't skewed by large values like the average.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.