Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am a beginner in statistics and probability study. Today I came across a new topic called the Mathematical Expectation. The book I am following says, expectation is the arithmetic mean of random variable coming from any probability distribution. But, it defines expectation as the sum of product of some data and the probability of it. How can these two (average and expectation) be same? How can the sum of probability times the data be the average of whole distribution?

Thanks in advance.

share|improve this question
up vote 31 down vote accepted

Informally, a probability distribution defines the relative frequency of outcomes of a random variable - the expected value can be thought of as a weighted average of those outcomes (weighted by the relative frequency). Similarly, the expected value can be thought of as the arithmetic mean of a set of numbers generated in exact proportion to their probability of occurring (in the case of a continuous random variable this isn't exactly true since specific values have probability $0$).

The connection between the expected value the arithmetic mean is most clear with a discrete random variable, where the expected value is

$$ E(X) = \sum_{S} x P(X=x) $$

where $S$ is the sample space. As an example, suppose you have a discrete random variable $X$ such that:

$$ X = \begin{cases} 1 & \mbox{with probability } 1/8 \\ 2 & \mbox{with probability } 3/8 \\ 3 & \mbox{with probability } 1/2 \end{cases} $$

That is, the probability mass function is $P(X=1)=1/8$, $P(X=2)=3/8$, and $P(X=3)=1/2$. Using the formula above, the expected value is

$$ E(X) = 1\cdot (1/8) + 2 \cdot (3/8) + 3 \cdot (1/2) = 2.375 $$

Now consider numbers generated with frequencies exactly proportional to the probability mass function - for example, the set of numbers $\{1,1,2,2,2,2,2,2,3,3,3,3,3,3,3,3\}$ - two $1$s, six $2$s and eight $3$s. Now take the arithmetic mean of these numbers:

$$ \frac{1+1+2+2+2+2+2+2+3+3+3+3+3+3+3+3}{16} = 2.375 $$

and you can see it's exactly equal to the expected value.

share|improve this answer
Thanks @Macro I can see now why is expectation the average. – Prakash Gautam Jun 13 '12 at 11:28
You're welcome @PrakashGautam, and welcome to the site! If you consider this answer definitive, please consider accepting it (no pressure at all, by the way, if you'd prefer to wait to see whether a better answer comes up) – Macro Jun 13 '12 at 12:13
since I am new to this great stack network I am not fully aware of such. Your answer though is definitely accepted. – Prakash Gautam Jun 13 '12 at 12:20
Wouldn't this be better illustrated by using the simpler set of {1,2,2,2,3,3,3,3}? The expression showing arithmetic mean of that set is identical to the expression showing the expectation value of that variable (if you convert the weighted products into simple sums). – Dancrumb Jun 13 '12 at 17:16
Re: "The expression showing arithmetic mean of that set is identical to the expression showing the expectation value of that variable (if you convert the weighted products into simple sums)" - Yes @Dancrumb, that was the entire point :) – Macro Jun 13 '12 at 17:16

The expectation is the average value or mean of a random variable not a probability distribution. As such it is for discrete random variables the weighted average of the values the random variable takes on where the weighting is according to the relative frequency of occurrence of those individual values. For an absolutely continuous random variable it is the integral of values x multiplied by the probability density. Observed data can be viewed as the values of a collection of independent identically distributed random variables. The sample mean (or sample expectation) is defined as the expectation of the data with respect to the empirical distribution for the observed data. This makes it simply the arithmetic average of the data.

share|improve this answer
+1. Good catch re: "The expectation is the average value or mean of a random variable not a probability distribution". I didn't notice this subtle misuse of terminology. – Macro Jun 13 '12 at 11:21

Let's pay close attention to the definitions:

Mean is defined as the sum of a collection of numbers divided by the number of numbers in the collection. The calculation would be "for i in 1 to n, (sum of x sub i) divided by n."

Expected value (EV) is the long-run average value of repetitions of the experiment it represents. The calculation would be "for i in 1 to n, sum of event x sub i times its probability (and the sum of all p sub i must = 1)."

In the case of a fair die, it is easy to see that the mean and the EV are the same. Mean - (1+2+3+4+5+6)/6 - 3.5 and EV would be:

prob x p*x

0.167 1 0.17

0.167 2 0.33

0.167 3 0.50

0.167 4 0.67

0.167 5 0.83

0.167 6 1.00

EV=sum(p*x) = 3.50

But what if the die were not "fair." An easy way to make an unfair die would be to drill a hole in the corner at the intersection of the 4, 5, and 6 faces. Further let's now say that the probability of rolling a 4, 5, or 6 on our new and improved crooked die is now .2 and the probability of rolling a 1, 2, or 3 is now .133. It is the same die with 6 faces, one number on each face and the mean for this die is still 3.5. However, after rolling this die many times, our EV is now 3.8 because the probabilities for the events are no longer the same for all events.

prob x p*x

0.133 1 0.13

0.133 2 0.27

0.133 3 0.40

0.200 4 0.80

0.200 5 1.00

0.200 6 1.20

EV=sum(p*x) = 3.80

Again, let's be careful and go back to the definition before concluding that one thing will always be "the same" as another. Take a look at how a normal die is set up and drill a hole in the other 7 corners and see how the EVs change - have fun.


share|improve this answer

The only difference between "mean" and "expected value" is that mean is mainly used for frequency distribution and expectation is used for probability distribution. In frequency distribution, sample space consists of variables and their frequencies of occurrence. In probability distribution, sample space consists of random variables and their probabilities. Now we know that total probability of all variables in sample space must be=1. Here in lies the basic difference. The denominator term for expectation is always =1. (i.e Summation f(xi) = 1) However no such restrictions on summation of frequency (which is basically total number of entries).

share|improve this answer

protected by whuber Jan 22 at 15:56

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.