Consider the words "banana" and "split". Assume that a pre-trained word embedding (say, word2vec GoogleNews) has the vectors like so:
banana_vec = array([[1.56, -2.46, 6.13, ... , -2.81]])
split_vec = array([[3.56, 9.45, -2.43, ... , 5.32]])
Now consider a completely new data set consisting of documents (sentences). Assume that one of the documents is the sentence "banana split".
How does one represent that sentence with the pre-trained word embeddings?
Things I've considered include:
Sum word embedding elements for each word
So this would give something like
bananana split = array([[v1, v2]])
v1 is the sum of elements in
v2 is the same for
L2 norm of word embedding elements for each word
So this would be the same, but the L2 norm instead of the sum.
Sum word embedding elements across words
So this would be:
banana split = array([[v1, v2, v3, ... , v100]])
v1 = 1.56 + 3.56
L2 norm of embedding elements across words
Same as above except
v1 = sqrt(1.56^2 + 3.56^).
Or is it something completely different?
Thanks in advance.