Strategy when introducing a new arm Let's say we have a bandit with two arms, and we know that one arm has a reward probability 0.5 and the other is unknown. How do we create a strategy to maximise the reward?
 A: Assume that you know which of the two arms has a known reward probability, and that without loss of generality the expected reward for arm $1$ is $\mu_1 = 0.5$ and you don't know the reward for arm $2$, i.e. $\mu_2$ is unknown. In this problem you have two (mutually exclusive) cases:
$(1)$ $\mu_1\geq \mu_2$,
$(2)$ $\mu_1 < \mu_2$
In case $(1)$ the optimal (horacle) strategy is the one always selecting arm $1$, while in case $(2)$ is the one that always selects arm $2$.
Of course, since $\mu_2$ is unknown, you need to estimate it by repeatedly sampling arm 2 and build an estimate. If $a_t$ is the arm selected at time $t$, and $r_t$ is the observed (noisy) reward
$$ 
\hat{\mu}_2(t) = \sum_{s = 1}^t r_s \mathbb{1}_{\{a_s = a_2\}}.
$$
Since $\mu_1$ is known, the optimal allocation would be the one sampling arm $2$ until you are confident about its estimation (e.g. you can obtain bounds on the probability that $\mu_1 > \mu_2$ through concentration inequalities) and then commit to the arm with the highest reward, i.e. $a_t = a_1$ if $\mu_1 \geq \mu_2$, and $a_t = a_2$ otherwise.
This doesn't really seem like a bandit problem since you never need to sample arm 1 but is more similar to hypothesis testing in which the hypothesis is $\mu_1 \geq \mu_2$.
If you do not know which of the arms has the known reward this post is related.
