# Do Bernoulli bandits need a different treatment if the rewards are sparse?

I have a problem where, effectively, my slot machines have very low payout probability (on the order of 1% for the "best" slot machines) and my goal is to minimize the number of actions to get one reward. I'm wondering if my goal is still equivalent to maximizing long term reward (as in the original formulation). At a higher level, I'm wondering which of the available tools for solving Bernoulli bandit are best to use in my scenario, or even if an altogether different approach would be more appropriate.

I should mention, I have an initial guesstimate for the reward probability of each slot machine (and they are distinct) and I can incorporate that into whatever method I use.

• You may as well say all the winning rewards are of equal value so wanting in the long term to minimise expected gaps is the same as wanting to maximise the expected number of rewards which becomes the same as wanting to maximise the expected amounts of rewards. Oct 6, 2023 at 15:00
• @Henry I was thinking about it this way too. I just felt a little unsteady about it for some reason. I think you saying it "out loud" helps. Oct 6, 2023 at 15:03
• But now I'm just wondering which is the best approach to use with sparse rewards. Oct 6, 2023 at 15:03

• Suppose you were planning to play a total of $n$ times and want to maximise the number of rewards you received. Would you initially cycle until the first reward and then stick with that arm forever? Oct 6, 2023 at 14:56