Mean, Average and Expectation What is the precise difference between the Mean, Average and Expectation ? Is there any mathematical formaulation to show the difference ? I see that often they are used interchangeably.
 A: There already is an answer on CV regarding the difference between mean and average, which actually ends at saying that, briefly, a mean is a particular case of average.
Whitout entering too much in details (since Wikipedia does it very well), here are some intuitions: 
An expected value is a theoretical average in the sense that you need to get all observations related to the phenomenon under question to compute it, which is rarely achievable, too costly, or simply impossible. This leads to approximate the expected value instead of really getting it. Note that the more data you have, the better its approximation, since this approximation is said, hopefully, to converge of the sample size:  this is the so-called Asymptotic law. Which commonly works very well, e.g. @TimAtreides' flipping coin perfectly illustrates this point.
And sometime, even if you have a big sample size, and that you can compute an average, the obtained value has no real sense, because the black box process which has generated your data is governed by a law of distribution which has simply no Expected value. Imagine, for example, that you have the following dataset
$S = \left[-112, 12, 30, -40, ..., 95 \right]$
And that the sample $S$ currently contains $n$ obervations. For sure, you will be able to compute the average. And since, as said above, the bigger $n$, the better the approximation, we may want to observe and add more values to this dataset. Say the next observed value is $10^{22}$. Your average is very likely to change radically. Let us observe and add another value, say $-10^{123}$, and another again, $10^{66}$, and so on. As you can see the numerical average does not converge/stabilize at all ! The data generating process we are trying to explain follows a distribution which has no expected value. A canonical example of this is the Cauchy distribution.
To summarize the intuition, the precise difference between Expectation and Average, is directly related to the one there is between what actually has a real sense (when trying to understand and explain the functionning of a data generative process) and what we can do/compute. Sometimes what one computes is a not so bad approximation of what is going out from the black box, sometimes not.
A: They are pretty much interchangeable.   Which you use and when may have different connotations, however.  And while almost everyone recognizes them as interchangeable, some people and textbooks do not agree.  Here are their connotations, as far as I know:


*

*Mean: The arithmetic mean.  That is, the sum divided by the total number of observations.  Not to be confused with the geometric mean.

*Average: Almost always the same as the mean.  Some people believe the term "average" to be the mean paired with its standard deviation.  Some people consider any measure of center to be an average.  (ie, "the median is an average").  But the people who use this context are small.  Don't know if I've ever met any in real life.

*Expectation: The connotation is that this is a theoretical mean.  For example, if you flip a coin 10 times, recording a 1 for heads and 0 for tails, you get 6 heads and 4 tails.  The mean will be 0.6, but the expectation would have been 0.5. 

