Skip to main content
Cross Validated

Return to Question

deleted 4 characters in body
Source Link
Ryan Zotti
Ryan Zotti
  • 6.8k
  • 6
  • 33
  • 36

Let's say you want to train a model so that you can make some predictions when you get some future data. You find some training data. Some of the training records have labels but other records do not. One approach might be to apply semi-supervised machine learning involving two separate models. The first model trains on only labeled training records. Then the first model makes predictions on the training records without labels. Finally, the you train a second model on all of the training data: records with real ground-truth labels (what the first model used for learning), and records that don't have original labels but that now have pseudo labels, which are the predictions from the first model.

Does the second model add any value? Will the second model provide meaningfully different (more accurate) predictions than the first the first model? Isn't the second model at best going to give the same (but over-confident) answers as the first model? Presumably a model can't outperform its ground truth (i.e., in a classification task 100% accuracy is the most accurate you can get). If a large portion of the second model's ground truth was generated from the first model, I don't see how the second model could be more accurate than the first model. That makes me wonder, why even bother with the second model?

Let's say you want to train a model so that you can make some predictions when you get some future data. You find some training data. Some of the training records have labels but other records do not. One approach might be to apply semi-supervised machine learning involving two separate models. The first model trains on only labeled training records. Then the first model makes predictions on the training records without labels. Finally, the you train a second model on all of the training data: records with real ground-truth labels (what the first model used for learning), and records that don't have original labels but that now have pseudo labels, which are the predictions from the first model.

Does the second model add any value? Will the second model provide meaningfully different (more accurate) predictions than the first the first model? Isn't the second model at best going to give the same (but over-confident) answers as the first model? Presumably a model can't outperform its ground truth (i.e., in a classification task 100% accuracy is the most accurate you can get). If a large portion of the second model's ground truth was generated from the first model, I don't see how the second model could be more accurate than the first model. That makes me wonder, why even bother with the second model?

Let's say you want to train a model so that you can make some predictions when you get some future data. You find some training data. Some of the training records have labels but other records do not. One approach might be to apply semi-supervised machine learning involving two separate models. The first model trains on only labeled training records. Then the first model makes predictions on the training records without labels. Finally, you train a second model on all of the training data: records with real ground-truth labels (what the first model used for learning), and records that don't have original labels but that now have pseudo labels, which are the predictions from the first model.

Does the second model add any value? Will the second model provide meaningfully different (more accurate) predictions than the first the first model? Isn't the second model at best going to give the same (but over-confident) answers as the first model? Presumably a model can't outperform its ground truth (i.e., in a classification task 100% accuracy is the most accurate you can get). If a large portion of the second model's ground truth was generated from the first model, I don't see how the second model could be more accurate than the first model. That makes me wonder, why even bother with the second model?

Source Link
Ryan Zotti
Ryan Zotti
  • 6.8k
  • 6
  • 33
  • 36

Semi-supervised learning with two models

Let's say you want to train a model so that you can make some predictions when you get some future data. You find some training data. Some of the training records have labels but other records do not. One approach might be to apply semi-supervised machine learning involving two separate models. The first model trains on only labeled training records. Then the first model makes predictions on the training records without labels. Finally, the you train a second model on all of the training data: records with real ground-truth labels (what the first model used for learning), and records that don't have original labels but that now have pseudo labels, which are the predictions from the first model.

Does the second model add any value? Will the second model provide meaningfully different (more accurate) predictions than the first the first model? Isn't the second model at best going to give the same (but over-confident) answers as the first model? Presumably a model can't outperform its ground truth (i.e., in a classification task 100% accuracy is the most accurate you can get). If a large portion of the second model's ground truth was generated from the first model, I don't see how the second model could be more accurate than the first model. That makes me wonder, why even bother with the second model?