I would like to know if UCB1 for multi armed bandit problems is deterministic or stochastic.
I understand that the arm chosen depends on the expected reward and the "width" of the upper bound, which depends on the number of times I have pulled that arm. So my guess is it's deterministic, but I'm not sure.
Also I am supposing my MAB is not dealing with adversarial rewards - is this the key to say it's deterministic?
- edit - I have since found that it should be stochastic, as opposed adversarial. How can I understand this though? Does this mean than adversarial is deterministic? I wouldn't have said so, the only difference is who/what is giving me my rewards, but I have the same knowledge in the two cases, I guess? Could you help clarify?