Expectation/Expected value is an operator that can be applied to a random variable. For discrete random variables (like binomial) with $k$ possible values it is defined as $\sum_i^k x_i p(x_i)$. That is, it's the mean of the possible values weighted by the probability of those values. Continuous random variables can be thought of as the generalization of this: $\int x dP$. The mean of a random variable is a synonym for expectation.
The Gaussian (normal) distribution has two parameters $\mu$ and $\sigma^2$. If $X$ is normally distributed, then $E(X)=\mu$. So the mean of a Gaussian distributed variable is equal to the parameter $\mu$.This is not always the case. Take the binomial distribution, which has parameters $n$ and $p$. If $X$ is binomially distributed, then $E(X)=np$.
As you saw, you can also apply expectation to functions of random variables so that for a gaussian $X$ you can find that $E(X^2)=\sigma^2+\mu^2$.
The Wikipedia page on expected values is pretty informative: http://en.wikipedia.org/wiki/Expected_value