I have loss history graphs that look exactly like yours from some of my early experiments with word embeddings. When I was getting started, I just wanted to make sure that I had implemented word2vec correctly, so I kept my vocabulary and my corpus small. (FYI, I'm not working with human language data, I'm working with protein sequences; but the problems are similar.)
I could achieve loss values that were trivially close to zero with a very small corpus. With a somewhat larger corpus, I would achieve non-zero but still very small loss values after a few epochs, then the loss history would go absolutely flat. Learning stopped.
Here's what I think was happening in my experiments: with very small samples, it is possible for your Embedding layer plus your decoder to memorize your corpus. If a word appears in multiple contexts, once all the context word vectors are in their optimal positions, you can't do any better. In support of my theory, when I enlarged my vocabulary and corpus, this behavior disappeared.
I'm going to offer an opinion, which I hope someone will correct if I am wrong: when you create an embedding, you are performing unsupervised learning. I don't think that the idea of overfitting exists or matters when you are creating an embedding. If perfect embedding representation of a large body of input data is achieved in a relatively small number of dimensions, that could be a good thing. Eventually the trained embedding will be used as a component of supervised models, and in that situation the normal overfitting issues will definitely be relevant.