Say I have a button in my website that I want to optimize the size given some metric. In order to use multi armed bandits for that, there are some things I would like to consider:

  • My metric is a concave function with respect to the button size
  • The button currently exists, and I would like to try similar sizes to it at first. As I'm confident the sizes I tested are leading to better results, I start to explore further sizes. For example, say my button is 50px wide. I start testing 40px and 60px. As I get more confident that 60px is leading to better results, I start to try 70px, and so on.
  • The goal of the last bullet is to not have to specify a limit size to explore and at the same time to not try crazy values from the beginning.

I've read about epsilon-greedy and thompson sampling. Both seem to be adaptable to my case, but I would like to know if there is already a stablished method for that


The method introduced in this paper by Flaxman el al. does a form of approximated gradient descent for this type of problem. Specifically, they consider the setting in which you're optimizing over a convex set, and you receive feedback at one point per iteration. Their method constructs an unbiased estimate of the gradient at each iteration, and they show that this scheme performs well.

There may be more recent developments in this space, but I like the elegance of this method.

I assume what you mean by a "natural ordering between bandits" is that the decision set is continuous and convex (the set of button sizes in your case), rather than a discrete set of objects.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.