Just wondering if any previous work compared semi-supervised learning vs supervised learning?
Currently, I have got both datasets with and without labeling. And therefore, it is intuitive for me to apply the semi-supervised learning scheme (using SVM based classifier) to do the classification.
I trained my SVM classifier using the labelled data. The training also involves a leave-one-out cross-validation to optimize the parameters of SVM. Then I applied the trained model to do the prediction for the unlabeled data.
Actually, it is very similar to the supervised learning, we can split the data for training, cross-validation and testing. The training and cross-validation processes are the same. The only difference is the testing data in supervised learning are with labels.
- Then does this mean that the semi-supervised learning can naturally prevent the overfitting of the classifier?
For example, if the classifier is overfitted in supervised learning, the training error is low, but testing error is high. Then in the semi-supervised learning, although I can't have quantitative validation for the unlabeled data, but some qualitative analysis can be done (for example, visually check the classified images), we can know that which setting is good during the training.
This seems a little bit odd to me, as I use the final predicted results to retrospectively choosing which parameter setting is the best one. Also, by doing this kind of qualitative analysis, I won't choose the settings that might overfit the classifier (Then does this mean that the semi-supervised learning can naturally prevent the overfitting of the classifier?)
So back to my main question, maybe very broad, but semi-supervised learning vs supervised learning, what are the benefits and limitations?
If I have got all the data with labels, I split them to training, cross-validation, and testing. And perform supervised learning and semi-supervised learning separately (both using same classifier like SVM). If the results are very similar, what does this mean? If the results differ to each other very much, what does this mean?