0
$\begingroup$

I have a two ML models model_a and model_b that optimize on an event, label_a.

  • I have a small volume of labels for model_a and a large volume of labels for model_b.
  • The features used in these models have a large amount of overlap, ~80%.
  • model_b is trained with labels that may only be 45-60% accurate in the context of model_a.

I'd like to use model_b label data to train model_a. I'm considering two approaches:

  • Use the labels directly for model_a, weighting the labels if their accuracy is known.
  • Use transfer learning to apply the learning from trained model_b to model_a before training on the smaller corpus of model_a labels

How should I be thinking about the tradeoffs between these two methods? Is one likely to be more successful than the other? Are there alternative options I've not considered?

$\endgroup$
  • 1
    $\begingroup$ you should provide more details about your models and their differences and analogies. also what's the difference and analogies between data sets of the two models? $\endgroup$ – carlo Nov 10 '19 at 21:04
0
$\begingroup$

This sounds to me like you would be just learning model B problem using model A, thus I don't see a benefit in learning wrong labels, even if they are weighted down. Transfer learning might work, be the concept is too broad and can mean many things.

If you want labels from model B to guide your model A predictions, you can include them as features, instead of faux outcome. Thus you can learn the relationship between the predicted label from model B and label A directly. Careful for double dipping tho.

Another possibility is to use semisupervised learning, to not to throw away unlabeled data, but that does not always work.

$\endgroup$
0
$\begingroup$

By saying you "have a small volume of labels for model_a" you mean that you have few samples in the model_a dataset or that you have many samples but only a few of them are labeled?

In the latter case another option could be programmatical labeling of the model_a samples with Snorkel:

https://www.snorkel.org/

$\endgroup$
  • $\begingroup$ Many samples, few labels. Unfortunately label expansion won't work for my dataset. $\endgroup$ – Ethereal Nov 15 '19 at 17:17
  • $\begingroup$ just out of curiosity: why it won't work? $\endgroup$ – Wassermann Nov 17 '19 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.