I'm working on implementing a multi-armed-bandit-like approach for determining the best price to offer for a product. Our goal is to optimize profit, meaning, we want to find the price where (price-cost) x number of purchases is greatest. The problem is that there is a known pattern of seasonality within the week. So, if I set the price to the same thing on a Monday vs. a Wednesday, they might have dramatically different volume.

That's my specific situation, but it started me wondering more broadly about using a multi-armed bandit when there are external factors (aside from which "arm" of the bandit you pull) that affects the results you get. Is there a way of taking these factors into account? Or-- perhaps there are results that suggest that if you run the algorithm long enough, it converges in spite of the other factors?

  • 1
    $\begingroup$ I like the way you talk about retail in terms of banditry... ;-) One thing to keep in mind is that price sensitivity (demand elasticity) will vary over the week. And if a bargain succeeds in drawing the Saturday buy-for-the-whole-week crowd, it may result in halo effects even for non-promoted items for people who do their whole week's shopping at the place that has the most attractive promotions on each particular Saturday. The whole topic is really much too big for a CrossValidated question (after all, there are people making their living doing just this - like me, for instance). $\endgroup$ – Stephan Kolassa Jan 28 '13 at 19:37
  • $\begingroup$ I definitely get that! I'm fine with starting out with a simple bandit and working my way up from there. Are there any papers you would recommend reading? $\endgroup$ – Denise Jan 28 '13 at 20:12
  • $\begingroup$ Sorry, nothing that really comes to mind... Good hunting! $\endgroup$ – Stephan Kolassa Jan 28 '13 at 20:17
  • $\begingroup$ There is a sizable body of work under the title "contextual bandits", which might be of interest to you. $\endgroup$ – Innuo Jan 28 '13 at 22:33
  • $\begingroup$ thanks! I think that's what I was looking for, just didn't know what it was called. $\endgroup$ – Denise Jan 29 '13 at 16:06

There is a sizable body of work under the title "contextual bandits", which might be of interest to you.

A bit more information. Contextual bandit algorithms attempt to solve the problem of picking the arm of a multi-armed bandit conditioned on some random vector (called the context). There is a discussion of the problem in this blog post. The traditional application is for internet advertising, where say the system sees a user's query (which is the context) and has to pick from one of K possible ads to show.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.