What class of problems does this belong to:

Similar to multi-armed bandit, but with a small adjustment/difference: by choosing an action at every time step you not only receive the reward of that particular action, but also realize what the reward would have been if another action was taken.

Has this problem been studied?


1 Answer 1


In the book Introduction to Multi-Armed Bandits by Aleksandrs Slivkins, the author distinguishes between 3 types of feedback that could be received by a bandit algorithm:

bandit feedback, when the algorithm observes the reward for the chosen arm, and no other feedback; full feedback, when the algorithm observes the rewards for all arms that could have been chosen; and partial feedback, when some information is revealed, in addition to the reward of the chosen arm, but it does not always amount to full feedback.

Here are some examples from the book for each case: enter image description here

So, what you are asking about may fall either under the partial or the full feedback cases.

  • $\begingroup$ and what algorithm would be able to deal with partial feedback? $\endgroup$
    – d56
    Sep 3, 2021 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.