# Euclidean distance with sparse and high dimension data

I have texts for a bunch of objects. From each text, I removed the stop words, and took each word as an attribute of the object. I then gave each word a rating based on sentiment analysis, so that the more positive the sentiment is the higher rating they receive.

I now want to compare the objects, using Euclidean Distance. A problem is that the length for the texts vary a lot, from a few sentences to a large amount. So, when I compare an object to a set of them, the ones that I think that should be closer end up further away, because they have way more data, so their ratings are way larger and the distances are too.

Is there a way to standardize the data among objects in the same set, considering objects with few data and objects with large amounts of it?

## 1 Answer

A much better similarity measure for sparse and high dimensional data is that of cosine similarity: $$s(x, y) = {x^Ty \over ||x|| ||y||}\\ = {\sum_i x_i y_i \over \sqrt{\sum_i x_i^2} \sqrt{\sum_i y_i^2}}$$ where $x$ and $y$ are constructed from their indivudal components, i.e. $x = (x_1, x_2, \dots, x_N$ for N dimensions. Each $x_i$ is what I call a component.

Some observations:

• if only one of the components is non-zero, the contribution to the distance will be $0$,
• $s(x, y) \in [-1, 1]$,
• $s(x, y) = 0$ if $x$ and $y$ are algebraically independent.
• $s(x, ax) = 0$ if $a > 0$.

In contrast to the Euclidean distance $||x - y||$:

• even if one of the components is zero, the contribution to the loss is unbounded,
• more general, $||x - y|| \in [0, \infty)$,
• $||x - ax|| > 0$ if $a > 0$.

Consequently, the cosine similarity cannot be dominated by single dimensions. Also, dimensions in which one of the vectors is zero have a rather "neutral" effect on the similarity. This can be quite beneficial in high dimensional, sparse vector spaces.

• Please expand your statements if only one of the components..., even one of the components...; and explain what is "component"? The thing is that what you say in the last paragraph doesn't seem to flow out of what you list above it transparently enough. – ttnphns Sep 30 '14 at 19:18
• I use components and dimensions interchangeably. I.e. $x_i$ is the i'th component/dimension of the vector $x$. (I added that to the answer.) – bayerj Sep 30 '14 at 19:29