2
$\begingroup$

I'm attempting to train a neural network to maximise a function, where the network takes a time series as input. At each time step it will make a decision based on what it thinks will maximise the reward function, however I have a small portion of the data which is labelled with a good action which would be valuable to incorporate.

Does anyone have any suggestions as to how to blend the two approaches?

I was thinking of either pretraining using the labelled data, or at each timestep calculating two weight updates - one which encourages matching the labels and another from gradient ascent on the reward function. I would weight these dependent on time, so that the update based on labelled data has more influence at the start of training and less towards the end. Any pointers/papers would be appreciated!

(combining supervised and unsupervised learning is the gist of what I'm looking for)

$\endgroup$
2
$\begingroup$

Combination of RL and supervised learning is well described in this paper. There is a lot of pointers in Related Work section as well.

In general, they combine the Double DQN approach with a supervised loss (they use a margin loss for that, but you can try to use cross-entropy, for instance). This allows them not only to learn the action values during training, but also to avoid behaving in an 'unusual' manner.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.