People despise using Eucliean distance in higher dimensional spaces because it is not a viable metric. People argue that the distance between two vectors becomes very large as the number of dimensions increases.
But to me: it makes sense as we add more dimensions that the Eucliean distance between two points in high dimensional space becomes larger as we add more dimensions. I don't quite see the problem here.
How does inflating the distance between two vectors invalidate the metirc itself? The magnitude of the distances may be larger, but why can't it still be a viable comparison metric?
The book elements of statistical learning gives a nice picture trying to describe why Eucliean distance fails:
Okay - so what? The distance between two points gets larger and larger? Why does that invalidate the metric? It makes perfect sense that the distance between these two vectors is larger as we add new dimensions because they are more dissimilar to one another in the newer dimensions.
Let's think of an example where we collected lengths of a bunch of toys:
The nearest point for a new toy 4.2 is 5.3 based on Euclidean distance. Now let's add another dimension called width of these toys.
- <5.3, 5.6>
- <2.2, 2.1>
- <1.2, 0.4>
An our new point is <4.2, 0>. Now the nearest point is <2.2, 2.1>. This makes sense. Because the second dimension is widely different there. People argue that distances become less meaningful. But I can still successfully apply it here and the resulting distance makes perfect sense to me.
Anyway I don't fully understand this hatred towards Euclidean distance - it seems to make perfect sense to me!