Expected value vs. most probable value (mode)

Question

The expected value of a distribution $f(x)$ is the mean, that is the weighted average value $$E[x]=\int_{-\infty}^{+\infty} x \, \, f(x) dx$$

The most likely value is the mode, that is the most probable value.

However do we expect somehow to see $E[x]$ a lot of times? Quoting from here:

If the outcomes $x_i$ are not equally probable, then the simple average must be replaced with the weighted average, which takes into account the fact that some outcomes are more likely than the others. The intuition however remains the same: the expected value of $x$ is what one expects to happen on average.

I cannot understand what does "happen on average" means, does this mean that, for istance, taking a measure a lot of time I expect to see $E[x]$ more than other values of $x$? But isn't this the definition of mode?

So how to interpret the statement? And what is the probabilistic meaning of $E[x]$?

---

I would also like to show an example where I do get confused. Studying $\chi^2$ distribution I learned that the mode is $\chi^2_{mode}=\nu-2$, while $E[\chi^2]=\nu$, where $\nu$ are the degrees of freedom of data.

I heard at university that, when doing a $\chi^2$ test after using Least Squares Method to fit a set of data, I should expect to get $\chi^2 \approx \nu$ because "that's what happens in general".

---

Did I misunderstand all of this or is the expected value somehow very probable? (Even if the most probable value is of course the mode)

Answer 1

For a Normal distribution, the expected value, a.k.a. the mean, equals the mode.

In general, not only is the expected value not only not the most likely (or at highest density), but it may have no chance of occurring. For instance, consider the random variable X which equals 0 or 2, each with probability 0.5. Then EX = 1, but the expected value,1, has 0 probability of occurring, while 0 and 2 are both modes of the distribution.

The quote "the expected value of x is what one expects to happen on average" is non-technical layman's language, which as evident by your confusion, only serves to confuse matters. Expected value has a very specific meaning in probability as being the mathematical average. Whereas in layman's language, an expected value or "on average" may be something expected to typically occur. These can be reconciled if "on average" is interpreted as being the mathematical average of what occurs.

Expectantly yours,

Joe Average

Answer 2

The expected value is a priori very abstract and there is no reason to think that it's the most probable outcome; as other have point out, it's easy to construct random variables for which $$P( X = E(X) ) = 0$$ (and the same with density if $X$ is continuos)

The only justification for the expected value, and the reason why we "expect to see it often", is the Law of large numbers:

if you have $n$ independent identically distributed variables $X_i$, then

$$\frac {X_1 + \dots + X_n}{n} \to E(X)$$

(for a suitable meaning of $\to$ which is pointless to investigate at the moment)

What does it mean? Imagine that you throw a coin with probability $p> \frac 12$ of landing head, which we will associate with the number $1$, and probability $1-p$ of landing tail (that is, $0$). What is the most probable outcome? 1! (that is, head) What is the expected value? $$E(X) = 1\cdot p + 0\cdot(1-p) = p$$

Now clearly "p" will never happen (it's either head or tail, either 0 or 1).

But launch the coin 10.000 times ,and record the times it came up head over the total number of throws. This number captures what we intuitively think of average ("average number of heads"). And the law of large numbers tells you that this number will be close to $E(X) = p$

Answer 3

I don't like the term "expected value" and didn't use it when teaching probability. "Arithmetic mean" is better, in my opinion, because the arithmetic mean of a 6-sided die is 3.5 but such a number doesn't occur. I did originally hear the term "expectation value" for the concept when in college. Lots of technical terms do not agree with the obvious non-technical meaning. ("Or" comes to mind.)

Note that a distribution may have more than one mode but the arithmetic mean is unique. Mode, mean, and median are different and have different uses.

Answer 4

The difference is easiest to see with discrete distributions:

Consider two sets of values where each number is equally likely to be drawn: {1,2,2,2,10} and {1,2,2,2,3}.

Both have the same mode (2), but the expected values differ. Expected value puts extra weight on large values while the mode simply looks for what value occurs frequently. So if you drew from this distribution a bunch of times, your sample average would be close to the expected value, while the most common integer to occur would be the close to the mode.

The mode is defined as $mode=arg{\max{f(x)}}$ while as you showed above, the expected value integrates over $x*f(x)$ so it considers the weight of each x.

The use of language to distinguish between different measure of central tendency is a common issue when learning statistics. For example, the median is another measure that isn't skewed by large values like the average.