Consider the words "banana" and "split". Assume that a pre-trained word embedding (say, word2vec GoogleNews) has the vectors like so:
banana_vec = array([[1.56, -2.46, 6.13, ... , -2.81]])
split_vec = array([[3.56, 9.45, -2.43, ... , 5.32]])
Now consider a completely new data set consisting of documents (sentences). Assume that one of the documents is the sentence "banana split".
How does one represent that sentence with the pre-trained word embeddings?
Things I've considered include:
Sum word embedding elements for each word
So this would give something like
bananana split = array([[v1, v2]])
where v1
is the sum of elements in banana_vec
, and v2
is the same for split_vec
.
L2 norm of word embedding elements for each word
So this would be the same, but the L2 norm instead of the sum.
Sum word embedding elements across words
So this would be:
banana split = array([[v1, v2, v3, ... , v100]])
where v1 = 1.56 + 3.56
L2 norm of embedding elements across words
Same as above except v1 = sqrt(1.56^2 + 3.56^)
.
Or is it something completely different?
Thanks in advance.