4
$\begingroup$

I'm working on implementing a multi-armed-bandit-like approach for determining the best price to offer for a product. Our goal is to optimize profit, meaning, we want to find the price where (price-cost) x number of purchases is greatest. The problem is that there is a known pattern of seasonality within the week. So, if I set the price to the same thing on a Monday vs. a Wednesday, they might have dramatically different volume.

That's my specific situation, but it started me wondering more broadly about using a multi-armed bandit when there are external factors (aside from which "arm" of the bandit you pull) that affects the results you get. Is there a way of taking these factors into account? Perhaps there are results that suggest that if you run the algorithm long enough, it converges in spite of the other factors?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I like the way you talk about retail in terms of banditry... ;-) One thing to keep in mind is that price sensitivity (demand elasticity) will vary over the week. And if a bargain succeeds in drawing the Saturday buy-for-the-whole-week crowd, it may result in halo effects even for non-promoted items for people who do their whole week's shopping at the place that has the most attractive promotions on each particular Saturday. The whole topic is really much too big for a CrossValidated question (after all, there are people making their living doing just this - like me, for instance). $\endgroup$
    – Stephan Kolassa
    Commented Jan 28, 2013 at 19:37
  • $\begingroup$ I definitely get that! I'm fine with starting out with a simple bandit and working my way up from there. Are there any papers you would recommend reading? $\endgroup$
    – Denise
    Commented Jan 28, 2013 at 20:12
  • $\begingroup$ Sorry, nothing that really comes to mind... Good hunting! $\endgroup$
    – Stephan Kolassa
    Commented Jan 28, 2013 at 20:17
  • $\begingroup$ There is a sizable body of work under the title "contextual bandits", which might be of interest to you. $\endgroup$
    – Innuo
    Commented Jan 28, 2013 at 22:33
  • $\begingroup$ thanks! I think that's what I was looking for, just didn't know what it was called. $\endgroup$
    – Denise
    Commented Jan 29, 2013 at 16:06

2 Answers 2

Reset to default
5
$\begingroup$

There is a sizable body of work under the title "contextual bandits", which might be of interest to you.

A bit more information. Contextual bandit algorithms attempt to solve the problem of picking the arm of a multi-armed bandit conditioned on some random vector (called the context). There is a discussion of the problem in this blog post. The traditional application is for internet advertising, where say the system sees a user's query (which is the context) and has to pick from one of K possible ads to show.

Improve this answer
$\endgroup$
0
0
$\begingroup$

The setting you describe resembles non-stationary bandits. In this setting, the means of the arms are allowed to vary over time.

The following papers might be of interest:

  1. A Linear Bandit for Seasonal Environments
  2. Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
  3. On Slowly-varying Non-stationary Bandits

Here there are also have these lecture notes which seem relevant.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.