Intuitive meaning of expected value

Question

Wikipedia gives the intuitive meaning of the expected value as

In probability theory, the expected value of a random variable, intuitively, is the long-run average value of repetitions of the same experiment it represents

It seems that the "long-run average value of repetitions" should be likely to occur in an experiment. I would expect $P(X=E(X))$ to be very high. After all, if a value is the long-run average value of repetitions, then it seems to occur frequently.

However, for example, $X \sim B(n=20, p=0.05)$. $E(X) = 1$, but $P(X=1) = 0.377$.

Why is "the long-run average value of repetitions" not likely to occur in an experiment?

Answer 1

There are even worse scenarios: let $X \sim \mathcal N(0,1)$ be a standard normal random variable. Then $\mathbf E X = 0$ and $\mathbf P (X = \mathbf E X) = 0$! This will hold for continuous random variables in general, but might also hold for discrete random variables, for example if $X \sim \operatorname{Binom}(\frac 1 2, 1)$.

That the expected value can be seen as the "long-run average value of repetitions" is not the usual characterization of the expected value, this phenomenon is described by the law of large numbers.

Another way to think about the expected value is as the center of mass of your probability distribution. The center of mass is determined by the distribution as a whole, so you do not necessarily need a high amount of mass around the center of mass. Think about a seesaw with equal masses on each ends, then the center of mass is in the middle of the seesaw, but there is no mass in the neighborhood of the center of mass.

Answer 2

By "long-run average" they mean "if you repeated this experiment many, many times". Assuming that the quantity of interest is a random variable, then the outcome of the experiment would be just a single draw from the distribution of the possible outcomes. It is random, so it can be anything within the support of the variable.

I would expect $P(X=E(X))$ to be very high.

Why? Not at all. If the variable of interest is continuous, then $\Pr(X=x) = 0$ for any $x$. Another example is a discrete random variable, it can have a floating-point expected value, that would be impossible to observe among the samples.

You also can check the Why is the expected value named so? thread, as it seems to be related to your question.