5
$\begingroup$

I have an existing set of data and plan to generate more data that follows the same pattern. To do this, I plan to use unsupervised learning. How can I provide feedback on the generated data and reinforce "good" and discourage "bad" results?

In other words: how can I combine reinforced and unsupervised learning? Or am I approaching this problem the wrong way?

edit: I am interested in using machine learning to generate music. It would be comparatively easy to use an unsupervised network to generate new music, but how would human graders come into play? Let's say I generate 10 samples and have a human answer good or bad, how can I improve the network? I thought of simply feeding back the "good" pieces back in, but there must be a better way.

Improve this question
$\endgroup$
4
  • $\begingroup$ Are you thinking of human input that says "no, these samples are no good," or some kind of automated mechanism? $\endgroup$
    – Danica
    Commented Nov 6, 2016 at 22:16
  • $\begingroup$ @Dougal It would be human input $\endgroup$
    – Streetlamp
    Commented Nov 6, 2016 at 22:23
  • $\begingroup$ This is awfully broad. Why don't you just generate data from a fitted distribution w/ the same parameters, & correlation matrix? $\endgroup$
    – gung - Reinstate Monica
    Commented Nov 6, 2016 at 22:45
  • 1
    $\begingroup$ @gung I have edited the question to be more specific $\endgroup$
    – Streetlamp
    Commented Nov 6, 2016 at 23:06

1 Answer 1

Reset to default
2
$\begingroup$

Something that reminds me of this is the framework of generative adversarial networks. These models have two parts:

  • a generator, which is usually implemented as a neural network mapping uniformly distributed inputs to the data you generate, e.g. images, and
  • a discriminator, which tries to classify between generated samples and true data.

The generator is updated to try to trick the discriminator (by gradient descent to minimize its accuracy), which is then updated again to deal with the new samples.

This framework has been very popular in the past few years, but is very tricky to use in practice, and there's a lot of ongoing research (e.g. this paper from earlier this year and this one just published two days ago) in getting them to work more reliably.

I'm not aware of anyone having put humans into the loop yet. You could imagine, maybe, having human annotators do some kind of label smoothing for the discriminator: true data set samples get the label 1, terrible samples get label -1, okay ones get -.75, great ones get 0, maybe the best ones get labeled as .5 or even 1. I haven't thought through the consequences of this too much, but it might help.

As an aside, incorporating human feedback into machine learning is often termed active learning.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.