I've implemented the word2vec algorithm according to its negative sampling architecture,using a shallow neural network that performs binary classification on word-embedding vector pairs. The network is expected to output 1 for pairs that occur in the corpus and 0 for the random negative pairs. In the final neuron, my implementation calculates the dot product of the two vectors before passing it to a sigmoid activation and later down the line, cross entropy is calculated and averaged over the batch.
My question is, should I use cosine instead of dot? I'm well aware that they differ only by the normalization of the vectors, however, I was unable to find a clear answer. Does this actually affect the quality of the embedding vectors? Or the use of cosine vs dot only matters when similarity is calculated for among the trained embedding vectors?