# Multi-armed bandit algorithms in Java?

Multi-armed_bandit problem defenition from Wikipeda:

"In probability theory, the multi-armed bandit problem (sometimes called the K- or N-armed bandit problem) is the problem a gambler faces at a row of slot machines, sometimes known as "one-armed bandits", when deciding which machines to play, how many times to play each machine and in which order to play them. When played, each machine provides a random reward from a distribution specific to that machine. The objective of the gambler is to maximize the sum of rewards earned through a sequence of lever pulls" (http://en.wikipedia.org/wiki/Multi-armed_bandit)

In other words Multi-armed bandit (MAB) problems are a class of sequential resource allocation problems concerned with allocating one or more resources among several alternative (competing) projects / 'arms'. Such problems have a conﬂict between making decisions (allocating resources) that yield high current rewards, versus making decisions that sacriﬁce current gains with the prospect of better future rewards.

Any Java implementations for these algorithms?

• Most certainly. What is your goal? Apr 11, 2013 at 11:31
• The goal is to learn MAB using Feynman's problem as a toy task: feynmanlectures.info/solutions/restaurant_problem_sol_1.html Apr 11, 2013 at 12:08
• If you want to learn, why not code it yourself? Apr 11, 2013 at 12:30
• Other people code helps me to understand how the algorithm works. Do you have any examples of MAB in Java or Python, maybe? Apr 11, 2013 at 13:03
• This is probably what you are looking for: mloss.org/software/view/415
– BigG
Jan 28, 2014 at 18:50

Play machine $j$, that maximizes
$$\bar{x}_j+\sqrt{\frac{2\ln n}{n_j}},$$ where $\bar{x}_j$ is the average reward obtained from machine $j$, $n_j$ is how often $j$ has been played and $n$ is the number of total plays. The 2 in the formula is a constant which can be changed to tune the exploration vs exploitation of the algorithm (higher number results in more exploration).