For semi-supervised learning, is more pseudolabels always better than less pseudolabels? Let's say I have a labeled dataset $L$ and unlabeled dataset $U$, where $U \gg L$. Suppose I focus on a subset of $U$ called $u$ and generate a subset of $u$ I'll call $u_L$ that consists of pseudolabels for $u$ that I've generated using Tri-training. Suppose $U$ is so large, that depending on the amount of datapoints I can choose to make up $u$, I can make $u_L$ virtually as large or as small as I want, from $10$ datapoints to $100$ to $1000$ to $1000000$ if I so wished, all from generating the pseudolabels from tri-training.
If I want to eventually train a model using $L \cup u_L$, is there any reason I'd want to restrict the size of $u_L$? Generally speaking when it come to ML, the more representative data the better the model, but in this circumstance I'm not sure. I can't seem to find anything about this.
 A: Semi-Supervised Learning (SSL) while not admitting it, emphasises the need to monitor the learning procedure via learning curves. As such, while we do not actively restrict the size of $u_L$ it should "restrict itself"; this might happen because we cannot successfully label any new unlabelled data or because we have actual performance degredation. Saito et al. Asymmetric Tri-training for Unsupervised Domain Adaptation (2017) has a really nice investigation of this phenomenon (see Fig. 4) where we observed clear plateauing in performance. That's why you correctly note that neither the original tri-training reference by Zhou & Li (2005) Tri-Training: Exploiting Unlabeled Data Using Three Classifiers or other standard self-training references make any references to a "max-ratio". (Maybe you have already checked the short monograph Introduction to Semi-Supervised Learning (2009) by Zhu & Goldberg but I found it very readable and helped me have a more unified view of the subject). That said, from a practical perspective it makes sense to constantly test your model performance on real data too. Ultimately all the success stories of self-driving cars, sentiment analysis, etc. get a lot of real data feedback. As Oliver et al. (2018) say in Realistic Evaluation of Deep Semi-Supervised Learning Algorithms:
"SSL is most likely the right choice for practitioners: (...)
When the labeled dataset is large enough to accurately estimate validation accuracy, which is necessary when doing model selection and tuning hyperparameter".
Similarly, be aware that one of the classes might be easier to "soft label" and indirectly lead us to an imbalanced learning setting, French et al. (2017) Self-ensembling for visual domain adaptation walked into that one and had to correct it via modifying their loss functions.
