When to use dot-product as a similarity metric

Question

I'm trying to understand which similarity measure should be used in which situations.

• Roughly speaking, when should someone use the dot product to assess similarity between vectors?
• Roughly speaking, when should someone use the norm of the vector difference to assess similarity between vectors?
• What are the pros and cons of these approaches?

I'm not looking for a specific answer on which one I should use for a specific problem. I'm wondering more generally about when these approaches should be expected to work well.

Answer 1

Euclidean distance (norm of difference) and dot predict are proportional to each other, while they are not equal, but roughly the same.

After normalizing $a$ and $b$ such that$\|a\| = 1$ and $\|b\| = 1$, these three measures are related as:

Euclidean distance = $\| a - b \| = \sqrt{\| a \|^2 + \|b\|^2 - 2 a^Tb} =\sqrt{2 - 2 \cos(\theta_{ab})}$

Dot product = $\|a\|\|b\| \cos(\theta_{ab}) = 1 \cdot 1 \cdot \cos(\theta_{ab}) = \cos(\theta_{ab}) $

Cosine = $\cos(\theta_{ab})$

Thus, all three similarity measures are equivalent because they are proportional to $\cos(\theta_{ab})$.

This is also discussed in here and in the Vector space model: cosine similarity vs euclidean distance thread and Wikipedia. There was even an empirical evaluation by Qian et al (2004) concluding that

Through our theoretical analysis and experimental results, we conclude that EUD and CAD are similar when applied to high dimensional NN queries. For normalized data and clustered data, EUD and CAD becomes even more similar.

Both metrics are similar and there are no strong reasons to prefer one over another in general.