Does it make sense to multiply two embedding vectors?

Question

Many researchers are using neural network to infer embedding vectors for words, users, or items. Word embeddings, e.g., word2vec, allow people to calculate sum, average, and difference over embeddings.

So does it make sense to multiply two embeddings? For instance, one 200-d user embedding and one 200-d movies embedding. The miltiplication results in a new 200-d vector, which should be able to represent the interaction of the user and the movie. The new vector can be an input of any prediction model. Does it make sense?

Answer 1

Yes it does. Here you can find example of network that uses multiplication, among other methods, for combining embeddings. As described in my answer

element-wise product $u*v$, is basically an interaction term, this can catch similarities between values (big * big = bigger; small * small = smaller), or the discrepancies (negative * positive = negative) (see example here).

So it is perfectly reasonable way of combining weights, but often, as in above example, people use in parallel several different methods for combining them, to produce different kind of features for the model.

Answer 2

I started working with words vectors for several weeks. I suspect that in order to obtain a valid answer to something like: "what is blood color?" the network will handle better Vec(blood)*Vec(color) insted of Vec(blood)+Vec(color) before calculating the sinus with all database words. Alas, I didn't test it yet.

Some stop words should change the way we operate with vectors. For example "I want non american food" should be calculated as: Vec(eat)+Vec(food)-Vec(american)

My main problem with words vectors is how slow it is when you have to calculate millions of sinus with 300 dimensional vectors ... I didn't find a way to accelerate this.