I think bandit algorithms(such as multi-armed bandit algorithms) can be considered as online algorithms because they make decision and update the parameters as data arrives. However, I can't find any articles/posts that confirm this statement.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Multi-armed bandit is a problem, not algorithm, there are multiple algorithms for solving it. Depending on your solution, you could solve it in online, or offline fashion. For example, you could decide that for a thousand of rounds you gather data by playing randomly, than use this data to estimate the expected payoffs, and given the estimate play the best arm. This would be clearly offline solution. On another hand, you could use something like $\varepsilon$-greedy, or Thompson sampling, that are online algorithms, as they adapt to incoming data, rather than processing it all-at-once.
