As the title of the question suggests, I'm interested in training a BERT-like model (and then use it to make some experiments on text-similarity).
Question: Could you share some references on the process of training a BERT model from scratch?
- A good reference to understand what do I mean by 'training from scratch' is something similar to what is done for the complete encoder-decoder training section in the article: 'The Annotated Transformer'.
- I'm particularly interested in the conceptual understanding of how a BERT-like model is trained.
Thanks in advance!