I am reading this paper where the training has no labelled outputs.

Consider the figure:

enter image description here

Now there are no labelled outputs z. But they are still training it somehow.

I see that it is reinforcement learning, but then reinforcement learning works in a different scenario where there are actions. I guess here they are sampling from some distribution the z values for the purpose of training, but how can that work, you need to have the ground truth values to compute the error.

I am really not able to understand this method in the context of training in a supervised way.

Please explain how you could explain how reinforcement learning can be used as a substitute for missing labels in supervised setting.

  • $\begingroup$ I don't see the figure in the paper, can you please add the source? The paper does not focus much on reinforcement learning: from what I recall from yesterday's defense I think the reinforcement learning was just some interpretation. I guess one can view each z as some action, and just like in a typical reinforcement learning setting one only gets the final rewards, and not a direct score on each action. $\endgroup$ Jan 20, 2017 at 21:42
  • $\begingroup$ @FranckDernoncourt thanks for the explanation, each z as action (by sampling different z from the generator), like they keep generating z till they choose one? As they can't be sure of z being determinstic. The figure is from here github.com/taolei87/rcnn/tree/master/code/rationale . Please explain a bit more. I am trying to understand and am getting things a little bit now. Thanks. $\endgroup$
    – Sie Tw
    Jan 21, 2017 at 1:00
  • $\begingroup$ They do have labeled outputs in this paper. For the sentiment analysis task there are sentiment labels as ground truth labels. The reinforcement learning is used to select a section of the text that is useful for predicting the sentiment. $\endgroup$
    – Aaron
    Feb 2, 2017 at 20:13


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.