This answer may have a slightly more mathematical bent than you were looking for.
The important thing to recognize is that all of these means are simply the arithmetic mean in disguise.
The important characteristic in identifying which (if any!) of the three common means (arithmetic, geometric or harmonic) is the "right" mean is to find the "additive structure" in the question at hand.
In other words suppose we're given some abstract quantities $x_1, x_2,\ldots,x_n$, which I will call "measurements", somewhat abusing this term below for the sake of consistency. Each of these three means can be obtained by (1) transforming each $x_i$ into some $y_i$, (2) taking the arithmetic mean and then (3) transforming back to the original scale of measurement.
Arithmetic mean: Obviously, we use the "identity" transformation: $y_i = x_i$. So, steps (1) and (3) are trivial (nothing is done) and $\bar x_{\mathrm{AM}} = \bar y$.
Geometric mean: Here the additive structure is on the logarithms of the original observations. So, we take $y_i = \log x_i$ and then to get the GM in step (3) we convert back via the inverse function of the $\log$, i.e., $\bar x_{\mathrm{GM}} = \exp(\bar{y})$.
Harmonic mean: Here the additive structure is on the reciprocals of our observations. So, $y_i = 1/x_i$, whence $\bar x_{\mathrm{HM}} = 1/\bar{y}$.
In physical problems, these often arise through the following process: We have some quantity $w$ that remains fixed in relation to our measurements $x_1,\ldots,x_n$ and some other quantities, say $z_1,\ldots,z_n$. Now, we play the following game: Keep $w$ and $z_1+\cdots+z_n$ constant and try to find some $\bar x$ such that if we replace each of our individual observations $x_i$ by $\bar x$, then the "total" relationship is still conserved.
The distance–velocity–time example appears to be popular, so let's use it.
Constant distance, varying times
Consider a fixed distance traveled $d$. Now suppose we travel this distance $n$ different times at speeds $v_1,\ldots,v_n$, taking times $t_1,\ldots,t_n$. We now play our game. Suppose we wanted to replace our individual velocities with some fixed velocity $\bar v$ such that the total time remains constant. Note that we have
$$
d - v_i t_i = 0 \>,
$$
so that $\sum_i (d - v_i t_i) = 0$. We want this total relationship (total time and total distance traveled) conserved when we replace each of the $v_i$ by $\bar v$ in our game. Hence,
$$
n d - \bar v \sum_i t_i = 0 \>,
$$
and since each $t_i = d / v_i$, we get that
$$
\bar v = \frac{n}{\frac{1}{v_1}+\cdots+\frac{1}{v_n}} = \bar v_{\mathrm{HM}} \>.
$$
Note that the "additive structure" here is with respect to the individual times, and our measurements are inversely related to them, hence the harmonic mean applies.
Varying distances, constant time
Now, let's change the situation. Suppose that for $n$ instances we travel a fixed time $t$ at velocities $v_1,\ldots,v_n$ over distances $d_1,\ldots,d_n$. Now, we want the total distance conserved. We have
$$
d_i - v_i t = 0 \>,
$$
and the total system is conserved if $\sum_i (d_i - v_i t) = 0$. Playing our game again, we seek a $\bar v$ such that
$$
\sum_i (d_i - \bar v t) = 0 \>,
$$
but, since $d_i = v_i t$, we get that
$$
\bar v = \frac{1}{n} \sum_i v_i = \bar v_{\mathrm{AM}} \>.
$$
Here the additive structure we are trying to maintain is proportional to the measurements we have, so the arithmetic mean applies.
Equal volume cube
Suppose we have constructed an $n$-dimensional box with a given volume $V$ and our measurements are the side-lengths of the box. Then
$$
V = x_1 \cdot x_2 \cdots x_n \>,
$$
and suppose we wanted to construct an $n$-dimensional (hyper)cube with the same volume. That is, we want to replace our individual side-lengths $x_i$ by a common side-length $\bar x$. Then
$$
V = \bar x \cdot \bar x \cdots \bar x = \bar x^n \>.
$$
This easily indicates that we should take $\bar x = (x_i \cdots x_n)^{1/n} = \bar x_{\mathrm{GM}}$.
Note that the additive structure is in the logarithms, that is, $\log V = \sum_i \log x_i$ and we are trying to conserve the left-hand quantity.
New means from old
As an exercise, think about what the "natural" mean is in the situation where you let both the distances and times vary in the first example. That is, we have distances $d_i$, velocities $v_i$ and times $t_i$. We want to conserve the total distance and time traveled and find a constant $\bar v$ to achieve this.
Exercise: What is the "natural" mean in this situation?