2
$\begingroup$

Suppose I have two vectors, v1 and v2, from which I can calculate the angle between these two vectors as a measure of their "distance", using the arccos function, say. For example:

v1 = c(100,200, 500,600)    
v2 = c( 50, 30,  10,5)      
v3 = c( 10,  7,  30,40) 

# pairwise angles
acos( as.numeric((v1 %*% v2) / (norm_vec(v1) * norm_vec(v2))) ) * 180 / pi  # 66.8017
acos( as.numeric((v2 %*% v3) / (norm_vec(v2) * norm_vec(v3))) ) * 180 / pi  # 66.67337
acos( as.numeric((v1 %*% v3) / (norm_vec(v1) * norm_vec(v3))) ) * 180 / pi  # 8.061138

This kind of measure will give similar distances (angles) regardless of the magnitude of the elements in the vectors. For instance, the distance between v1 and v2 is 66.80, and the distance between v2 and v3 is similarly 66.67, but clearly the magnitude of v2 and v3 are more similar than v1, so I am thinking of a measure that will also take the "magnitude" into account when calculating the dissimilarity. In other words, dist(v1, v2) should be greater than dist(v2,v3), but this result is obtained by still using the pairwise angle idea. Thanks!

UPDATE

Thank you all for your replies! For the equivalence of Euclidean and angle distances, I use the same vectors as above to calculate the Euclidean distances:

# Euclidean distance
ed <- function(x1,x2)
  sqrt(sum((x1 - x2) ^ 2))
ed(v1,v2)  
# 790.9014
ed(v2,v3)  
# 61.26989  --> imply v2, v3 are "closer"
ed(v1,v3)  
# 761.4782  --> imply v1, v3 are "far apart"

As you can see, v1, v2 and v1, v3 have similar magnitude of distances in Euclidean case, but for arccosine, they are different (66.8 vs. 8.06). Is there anything particular that is revealed by the angle distance but not the Euclidean distance? I think the orientation information is emphasized in the angle distance.

$\endgroup$
  • 2
    $\begingroup$ Your code explicitly takes the magnitudes out and you are asking not to, so why not just leave out the division and use the dot products? However, it's questionable that this would qualify as a "distance." That brings up a more fundamental issue: why are you doing this? In what kind of analysis will these distances be used and how do you intend to interpret them? $\endgroup$ – whuber Oct 1 '13 at 19:03
  • $\begingroup$ @whuber: thanks for your reply! The inclusion of the denominator as a normalizing factor is to use the arccos function. I am thinking of an alternative way to "additionally" account for the magnitude based on this pairwise angle dissimilarity measure. Yes, it is not a distance in the strict sense, but for my application it should be fine. $\endgroup$ – alittleboy Oct 1 '13 at 19:26
  • 1
    $\begingroup$ Without knowing your application it's not possible to provide objective advice about a distance. My first point--and this still puzzles me--is that if you want a measure that accounts for magnitude, then (obviously) you don't use an arccosine. If you simply did not do all the extraneous calculations needed to eliminate the effect of the magnitudes, you would stop at computing the dot products (such as `v1 %*% v2'). I'm mystified about what you're trying to achieve. $\endgroup$ – whuber Oct 1 '13 at 19:30
  • 1
    $\begingroup$ @alittleboy, 1) the law of cosines is valid for euclidean space of any dimensionality. 2) Yes, euclidean distance is far not ubiquitous. Tons of different other distance measures exist. You might use even the angle measured in degrees or radians (which is arc distance) and multiply it by the magnitudes. You are free to tailor any measure which you think best suits your tasks. $\endgroup$ – ttnphns Oct 1 '13 at 22:56
  • 2
    $\begingroup$ Even in infinitely many dimensions, any two vectors determine a subspace of dimension at most $2$: therefore the (Euclidean) relationships that hold in two dimensions among pairs of vectors hold entirely without any change at all in any number of higher dimensions, too. $\endgroup$ – whuber Oct 2 '13 at 15:23
7
$\begingroup$

There is a close relationship between Euclidean distance and Angular distance.
(see also: http://en.wikipedia.org/wiki/Cosine_similarity#Properties)

So if you want to take magnitude into account, you may actually be looking for Euclidean distance...

Let's look at squared Euclidean, for simplicity: $$ \sum_i (a_i - b_i)^2 = \sum_i \left(a_i^2 - 2 a_i b_i + b_i\right)\\ = \sum_i a_i^2 + \sum_i b_i^2 - 2 \sum_i a_i b_i = ||A|| + ||B|| - 2 (A \cdot B) $$

Now let's assume $||A||=||B||=1$, i.e. vectors standardized to unit length.
Then Euclidean distance is $\sqrt{2 - 2 (A\cdot B)}$!

In essence, Cosine similarity is like (squared) Euclidean distance after scaling each vector to unit length. I.e. if you have data that is very different in magnitude, but you do not want to take magnitude into account, then use cosine.

Note that angular similarity is scale invariant: $$ \operatorname{CosSim}\left(\frac{A}{||A||}, \frac{B}{||B||}\right) = \operatorname{CosSim}\left(A, B\right) $$ And if I didn't screw up somewhere (please edit then!): $$ \operatorname{Euclidean}\left(A,B\right)^2 = ||A||^2 + ||B||^2 - 2 ||A|| ||B|| \operatorname{CosSim}\left(A, B\right) $$

(Law of Cosines).

Conversely, normalizing your data before using Euclidean yields: $$ \operatorname{Euclidean}\left(\frac{A}{||A||}, \frac{B}{||B||}\right)^2 = 2\left[1 - \operatorname{CosSim}\left(A, B\right)\right] $$ (Note that the right hand is a popular way of converting cosine similarity to a distance!)
And cosine similarity is monotone to squared Euclidean on the normalized vectors: $$ \operatorname{CosSim}\left(A, B\right) = 1-\frac{1}{2}\operatorname{Euclidean}\left(\frac{A}{||A||}, \frac{B}{||B||}\right)^2 $$

As per @whuber's comment: the formula $$ \operatorname{acos}\left[\operatorname{CosSim}\left(A, B\right)\right] $$ yields the geodetic distance on the surface of the unit sphere to get from the point $A/||A||$ to the point $B/||B||$. For obvious reasons, the $0$ vector yields undefined results.

$\endgroup$
  • $\begingroup$ @Anoy-Mousse: thanks! Please see my updates for some questions related to your solution :) $\endgroup$ – alittleboy Oct 2 '13 at 16:05
  • $\begingroup$ Well, if you turn that around, Angular distance is essentially equivalent to Euclidean distance, when you take magnitude not into account... It's Euclidean distance with built-in normalization to unit length vectors. $\endgroup$ – Anony-Mousse Oct 2 '13 at 16:38
  • 1
    $\begingroup$ @Anony That's almost correct: the angular distance is the geodetic distance on the unit sphere. It is not a Euclidean distance. $\endgroup$ – whuber Oct 2 '13 at 16:45
  • $\begingroup$ "equivalent" is a broad term. There exists a formula to convert. ;-) I've been mostly using cosine similarity actually; not cosine distance, because there is more than one "angular distance". $\endgroup$ – Anony-Mousse Oct 2 '13 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.