I have an existing set of data and plan to generate more data that follows the same pattern. To do this, I plan to use unsupervised learning. How can I provide feedback on the generated data and reinforce "good" and discourage "bad" results?

In other words: how can I combine reinforced and unsupervised learning? Or am I approaching this problem the wrong way?

edit: I am interested in using machine learning to generate music. It would be comparatively easy to use an unsupervised network to generate new music, but how would human graders come into play? Let's say I generate 10 samples and have a human answer good or bad, how can I improve the network? I thought of simply feeding back the "good" pieces back in, but there must be a better way.

  • $\begingroup$ Are you thinking of human input that says "no, these samples are no good," or some kind of automated mechanism? $\endgroup$ – Dougal Nov 6 '16 at 22:16
  • $\begingroup$ @Dougal It would be human input $\endgroup$ – Streetlamp Nov 6 '16 at 22:23
  • $\begingroup$ This is awfully broad. Why don't you just generate data from a fitted distribution w/ the same parameters, & correlation matrix? $\endgroup$ – gung - Reinstate Monica Nov 6 '16 at 22:45
  • 1
    $\begingroup$ @gung I have edited the question to be more specific $\endgroup$ – Streetlamp Nov 6 '16 at 23:06

Something that reminds me of this is the framework of generative adversarial networks. These models have two parts:

  • a generator, which is usually implemented as a neural network mapping uniformly distributed inputs to the data you generate, e.g. images, and
  • a discriminator, which tries to classify between generated samples and true data.

The generator is updated to try to trick the discriminator (by gradient descent to minimize its accuracy), which is then updated again to deal with the new samples.

This framework has been very popular in the past few years, but is very tricky to use in practice, and there's a lot of ongoing research (e.g. this paper from earlier this year and this one just published two days ago) in getting them to work more reliably.

I'm not aware of anyone having put humans into the loop yet. You could imagine, maybe, having human annotators do some kind of label smoothing for the discriminator: true data set samples get the label 1, terrible samples get label -1, okay ones get -.75, great ones get 0, maybe the best ones get labeled as .5 or even 1. I haven't thought through the consequences of this too much, but it might help.

As an aside, incorporating human feedback into machine learning is often termed active learning.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.