There are several different measures of "location" or "central tendency". Expected value is the most popular one, but there are others -- median, mode, geometric mean, etc.
While all measures of central tendency are, in some ways, similar, it is important to remember that they actually measure different things. Here are the interpretations.
- Expected value (or mean). Expected value is useful for calculating the total when you have a large number of observations. Say you own a company with a large number ($N$) of employees. The expected salary is $E[X]$. The total salary that you have to pay out is $N E[X]$.
- Median. Median tells you about the typical observation. If you want to know what the typical person earns, look at the median salary, not the expected salary.
- Mode. Mode tells you the most likely outcome. If you are applying for jobs in a particular field, the modal salary is what you will most probably earn, not the expected salary.
There is confusion because, with symmetric distributions, which are common, mean, median, and mode are numerically equal. So people just think of the mean and not the median or mode. But, depending on the problem, what you are actually interested in could be the mode, even if it is numerically equal to the mean.
Now, let's look at your specific questions.
- Suppose each time you throw a die, you earn the amount of money that comes up. After throwing it a lot of times, you will earn $N E[X]$. The "fair price" that someone could charge you for making these throws is $N E[X]$.
- If you assign head and tail to numbers (think: amounts that you will earn or pay), then there certainly is an expected value. For $H = 0$, $T = 1$, $E[X] = 0.5$.