I want to train a neural network that is part of a multi-armed bandit problem. For each data sample, I have some features representing the context of the sample and there are x neurons in the output layer that represents the reward for this context for a specific action (each neuron represents an action). The issue is that for each sample, I know only the reward (the y value) for the action that was played. For all other actions, I don’t have the “ground truth” for them. What is the best practice in training my nn in that case?

  • $\begingroup$ You should learn about a subject called “reinforcement learning”. A good description of what to do exactly in the situation you describe above can be found here: karpathy.github.io/2016/05/31/rl . Essentially, you play for a while and take the actions actually sampled by the NN as “ground truth” in the loss function and then you multiply the gradient with the reward. Unfortunately, the n-armed bandit is a special situation where every episode is of length 1 (after only one action you start a new game) which makes some assertions look weird but the concepts are still valid... $\endgroup$ – Fabian Werner Jan 17 '19 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.