New answers tagged


Your argument is right. From the perspective of the language model, you have well-defined target labels and use supervise learning methods to teach the model to predict the labels. Calling it unsupervised pre-training is certainly sort of paper-publishing marketing, but it is not entirely wrong. It is unsupervised from the perspective of the downstream tasks....

Top 50 recent answers are included