Timeline for How to improve language model ex: BERT on unseen text in training?
Current License: CC BY-SA 4.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 4, 2022 at 15:55 | comment | added | IS92 | But I can't directly use the output of the multi-task fine-tuning because overall results are bad, I was thinking of some kind of ensemble but no sure how to design it, I would like to hear your thoughts on this :). | |
| Jan 4, 2022 at 15:55 | comment | added | IS92 | Yet, what I did is multi-task fine-tuning the LM by training on a very similar task (so I did not train on the original training set at all), it achieved an overall F-measure of approximately ~45%, however, sentences containing more specifically were misclassified 170 times only (compared when training on the original training data sentences containing more specifically were misclassified 750 times!! | |
| Jan 4, 2022 at 15:54 | comment | added | IS92 | Thanks Chris for your answer! Yes what I did is mainly data augmentation, using back-translation and also NLP augmenting tools like textAttack. Yet the problem that I don't want to use the sentences containing more specifically in my training data (or even its derivatives from back-translation and text attack, but supposedly I don't know the label, so I can't add it in my training data). | |
| Jan 3, 2022 at 14:35 | history | answered | chrishmorris | CC BY-SA 4.0 |