As discussed in the comments, this is not exactly a multi-armed bandit problem. In multi-armed bandit you know the rewards only after you "pull the arm" of your slot machine. For example, if you are running online ad campaign and you want to test between different ads, then you can only one add to user at a time and you don't know the rewards that you would get if showing different ads. This is why there is the exploration/exploitation trade-off: you can either explore "what would happen" by picking different arms, or exploit the one arm that is known to work best. Everything comes here at some cost and multi-armed bandit problem aims at suggesting the most optimal actions to balance those factors.
In your case, you have zero knowledge only before the first round, so first move can be done at random. After first move, you know all the rewards, so you can update the data on all arms simultaneously. Next, and the following steps, is to pick the arm that is currently known to give best rewards. You do not need the exploration step (i.e. randomizing the choices) since you have equal knowledge on all arms. The more steps you make, the better decisions you can make since you have more knowledge. You still can use something like randomized strategies to choose between arms, but all the further considerations is standard decision theory.
As a quick recap of decision theory, for each arm, you can expect some reward $r$ and we can define a preference, or utility, $U(r)$ of such reward, that defines your preferences over such rewards (e.g. utility of money is known to be non-linear). If two arms have different probability distributions $P_1$ and $P_2$ for the rewards, then the general approach would be to prefer to play the arm that gives us the greatest expected utility, e.g. when
$$
E_{P_1}[U(r)] < E_{P_2}[U(r)]
$$
then $P_2$ is preferred to $P_1$. Why do we care about expected value? This is nicely answered in the Why is the expected value named so? thread, in fact the whole idea of expected value emerged from gambling. You don't know $P_1$ and $P_2$, so you need to estimate expected value from the data, and you can use arithmetic mean for that.
Regarding your comments, it is true that with small samples your estimates of mean would be imprecise, but estimates of variance would be imprecise as well, and dividing one by another does not fix this. Moreover, you have equally imprecise information for all arms, since you saw the same number of samples for each. With the arm that has big variance, you could expect the the possible rewards to vary a lot, so they could be either very small, or very large. Your correction by dividing by variance seems to be a kind of hack to implement some sort of risk aversion in your utility function. This could, or could not, make sense in your particular application, but keep in mind, that this is your choice of what you consider as "acceptable". In standard multi-armed bandit problem, we randomize the choices to explore the space of possible rewards better, here you don't have this problem since you have equal knowledge on all arms, so it's pure exploitation.
