# How is addition and subtraction determined when using word embeddings like Word2Vec?

This question is particularly in the context of the word embedding algorithm Word2Vec.

I've noticed that many examples that are given in the original paper and other blog posts say things like vec(king) - vec(man) + vec(woman) = vec(queen) or vec(Madrid) - vec(Spain) + vec(France), but how is it determined when to use addition and when to use subtraction?

Intuitively, one can deduce that subtracting the country name from the capital and adding another country name would provide information on the new country's capital, but I was wondering if there was a formal explanation on how exactly this process goes.

There is not really a formal explanation, there is only intuition, like you have, backed up by actual computations that confirm it.

It just turns out that if you create a word embedding, that these computations work.

So, hypothetically, if 'Madrid' is mapped to [2, 3, 4], and 'Spain' is embedded as the vector [1, 1, 1], then the difference between those, [1, 2, 3], is very close to the difference between the embeddings of 'France' and 'Paris'. So you could say that the direction of [1, 2, 3], in the vector space, corresponds to the 'is the capital of' relation.

More realistically, these vectors are in 300d, not 3d as in the example above.

• Thank you for the answer. It is a bit of a disappointment that there is no formal definition. That makes it even more amazing that the authors were able to come up with that idea. :)
– Sean
Commented Jul 9, 2019 at 7:33
• Yes, it's quite an amazing result. Imagine hoping to find such a relation and then actually being able to publish it..
– Gijs
Commented Jul 9, 2019 at 11:13
• +1. Two other interesting tidbits: 1) you’re training a model to predict and it’s not very good, but it’s coefficients are the near-magical embedding, and 2) it’s based entirely on context and can lead to “synonyms” that aren’t really synonyms but rather are used in the same context. Commented Jul 9, 2019 at 14:11
• Commented Aug 19, 2019 at 7:29