I have a task for which I am using word embeddings but because limited data, I can't train. For one part of my task Word2Vec model of Google News is working okay while for the other one, Glove embeddings are working fine. I tried to merge these two by taking the vector of the word from both models and averaging it. However, that seemed to have changed the entire scenario and the results are pretty much random.
One explaination that comes to my mind is that latent features learnt by both models are pretty much different and hence, averaging these two is resulting in a random embedding of words.
Am I right? Is there any other way to merge embeddings from two sources?