Let's say I have a large corpus of documents. Instead of using a pretrained embedding model, I train my own non-contextual embedding model like w2v/fasttext from scratch on all the documents and save the model. Now I take a subset of my corpus for training a classification task with a sophisticated neural network. I tokenize every document and substitute the word vectors from my embedding model and train on this.
Since the word vectors for both train and test sets come from the same embedding model, can we say that the neural network has already somewhat seen the test set? Would it be cheating? Or is it that just the co-occurrence statistics used for learning the embeddings aren't complex enough for the model to be able to cheat at the specific classification task?