# Multi-armed bandit algorithms in Java?

Multi-armed_bandit problem defenition from Wikipeda:

"In probability theory, the multi-armed bandit problem (sometimes called the K-[1] or N-armed bandit problem) is the problem a gambler faces at a row of slot machines, sometimes known as "one-armed bandits", when deciding which machines to play, how many times to play each machine and in which order to play them. When played, each machine provides a random reward from a distribution specific to that machine. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls" (http://en.wikipedia.org/wiki/Multi-armed_bandit)

In other words Multi-armed bandit (MAB) problems are a class of sequential resource allocation problems concerned with allocating one or more resources among several alternative (competing) projects / 'arms'. Such problems have a conﬂict between making decisions (allocating resources) that yield high current rewards, versus making decisions that sacriﬁce current gains with the prospect of better future rewards.

Any Java implementations for these algorithms?

• Most certainly. What is your goal? – ziggystar Apr 11 '13 at 11:31
• The goal is to learn MAB using Feynman's problem as a toy task: feynmanlectures.info/solutions/restaurant_problem_sol_1.html – Anton Ashanin Apr 11 '13 at 12:08
• If you want to learn, why not code it yourself? – ziggystar Apr 11 '13 at 12:30
• Other people code helps me to understand how the algorithm works. Do you have any examples of MAB in Java or Python, maybe? – Anton Ashanin Apr 11 '13 at 13:03
• This is probably what you are looking for: mloss.org/software/view/415 – BigG Jan 28 '14 at 18:50

Play machine $j$, that maximizes
$$\bar{x}_j+\sqrt{\frac{2\ln n}{n_j}},$$ where $\bar{x}_j$ is the average reward obtained from machine $j$, $n_j$ is how often $j$ has been played and $n$ is the number of total plays. The 2 in the formula is a constant which can be changed to tune the exploration vs exploitation of the algorithm (higher number results in more exploration).