# Two armed bandit with a known expectation

Assume a two armed case with bernoulli rewards. We know that UCB1 gives a pretty tight bound for multiarmed bandit cases. What if we know the mean of one arm, how can we obtain a better strategy/algorithm than the UCB1?

The way I see it is that since we already know what to expect for one arm, we never explore it. So one would have to find the optimal amount of exploration for the unknown arm before we decide on one arm and exploit that forever after?

• Do you mean that we assume the knowledge of the exact expected outcome for choosing one of the arms? – Karel Macek Oct 12 '17 at 16:59
• One more question: do you consider finite horizon or infinite one? Do you consider discount factor $\gamma<1$? – Karel Macek Oct 12 '17 at 17:00
• I don't think it will change your big-O regret, since you'll still do half the exploring you would normally do. – combo Nov 2 '17 at 17:16

Long story short, in the two arms case, if you know the value of the two arms but don't know which one is which you can have constant regret (with respect to horizon $$T$$).
My educated guess is that only arm 2 is informative. So , if explorative pulls happens to be the best, then they are harmless (even if you don't know they are). You may want to use the index $$i_1 = \mu_1$$ $$i_2 = ucb_2$$
with $$\mu_1$$ the known value of arm $$1$$ and $$ucb_2$$ the classic ucb index of arm 2. If arm 1 is the best, the exploration bonus on arm 2 will cause classical logarithmic rate. However, if arm 2 is the best then your ucb will be at some point with high probability above $$\mu_1$$ and you will stop pulling arm 1 (and hence doing mistake) forever.