I want to assess distances between pairs of high-dimensional vectors (~1500 features). My vectors have been normalized by their L2 norms, so they all have unit length and point to the surface of a hypershpere.

I know of a particular instance of the "curse of dimensionality", which says that with increasing dimensions the difference between minimum and maximum distances tends to zero, and Euclidean distance becomes a useless metric. However, does this problem also hold in the particular case of normalized vector lengths? I've seen discussions noting that the bulk of the volume in high-dimensional spaces in concentrated in the outer shell, which is precisely the situation I'm dealing with. Does Euclidean distance somehow become "valid" again?

Alternatively, I would say that the distance between two vectors touching the surface of a hypersphere is simply theta/2*pi*r, where theta is the angle between vectors in radians, and the radius r is 1. (Consider a 2D circle where the only way from point A to B is by traversing the surface of the circle.) However, here too I am uncertain whether this simple intuition carries over to higher dimensions or whether unexpected behavior occurs.

  • 2
    $\begingroup$ The dimension of a hypersphere in 1500 dimensions is 1499. If you think that would be a useful reduction in dimensions then you have nothing to worry about :-). $\endgroup$ – whuber Aug 20 '15 at 20:57
  • $\begingroup$ I understand the L2 normalization only yields a dimension reduction of 1. But my goal is not to reduce dimensions. Rather, the normalization is a procedure that makes (conceptual) sense for my data, as does looking at the distance between vectors "on" the hyperplane. I'm new to high-dimensional thinking so I'm not sure what you're implying here. Are you saying that going from 1500 to 1499 dimensions won't (noticeably) affect these metrics and I might as well stick with 1500? Or that distances on hyperplanes are/aren't a good idea in general? $\endgroup$ – mrroy Aug 23 '15 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.