12
$\begingroup$

I'm studying Reinforcement Learning, and have come across multi-armed bandits.

Why are these called bandits? And why are they armed?

$\endgroup$
0

2 Answers 2

13
$\begingroup$

This is actually explained on the Wikipedia page

This is a classic reinforcement learning problem that exemplifies the exploration–exploitation tradeoff dilemma. The name comes from imagining a gambler at a row of slot machines (sometimes known as "one-armed bandits"), who has to decide which machines to play, how many times to play each machine and in which order to play them, and whether to continue with the current machine or try a different machine.

they even have a picture of few of those:

enter image description here

As noticed by Henry in the comment, there is even more accurate image on Wikipedia to show the etymology:

enter image description here

$\endgroup$
3
  • 6
    $\begingroup$ It's probably worth noting that in the 'old days' mechanical slot machines used to have a long handle that the user pulled to play. That was the 'arm'. These were even commonly included on machines after they were no longer necessary for operation but modern machine rarely have these which is why there are none visible in the image. $\endgroup$
    – JimmyJames
    May 18, 2020 at 21:42
  • 9
    $\begingroup$ And it's a one-armed bandit because it takes all of your money. $\endgroup$
    – hobbs
    May 19, 2020 at 0:37
  • 1
    $\begingroup$ See commons.wikimedia.org/wiki/… for a picture of four antique one-armed bandits and commons.wikimedia.org/wiki/… for some dressed up as bandits $\endgroup$
    – Henry
    May 19, 2020 at 8:54
5
$\begingroup$

In section 2.1 of Sutton and Bardo's Reinforcement Learning: An Introduction, they say:

[...] the k-armed bandit problem, so named by analogy to a slot machine, or “one-armed bandit,” except that it has k levers instead of one. Each action selection is like a play of one of the slot machine’s levers, and the rewards are the payout for hitting the jackpot.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.