Skip to main content
added 142 characters in body
Source Link
  1. Cosine is equivalent to (squared) Euclidean distance on L2 normalized data. So spherical k-means is using a distance. But it's actually using squared Euclidean distance, which does not satisfy the triangle inequality (and matematicians will kill kittens every time you call it a distance). While the nearest center is the same with respect to squared Euclidean and non-squared Euclidean, the optimum center is not. The optimum k-means result is usually not the least-sum-of-distances assignment (but the least-sum-of-squared-deviations).

  2. k-means cannot be used with arbitrary distances. Technically, it minimizes variance not distance... Study the convergence proof to understand why it does not converge with aebitrary metrics. It requires Bregman divergences.

  3. other algorithms may have other requirements. For example single-link clustering and DBSCAN do not require triangle inequality. DBSCAN does not even require symmetry (and single-link would implicitly use $\min\{ d(x,y), d(y,x)\}$). Other clustering algorithms do not use any distance at all. So your question title is bad. Your question is solely about k-means, and even an incorrect interpretation of k-means as minimizing distance.

  1. Cosine is equivalent to (squared) Euclidean distance on L2 normalized data. So spherical k-means is using a distance. But it's actually using squared Euclidean distance, which does not satisfy the triangle inequality (and matematicians will kill kittens every time you call it a distance). While the nearest center is the same with respect to squared Euclidean and non-squared Euclidean, the optimum center is not. The optimum k-means result is usually not the least-sum-of-distances assignment (but the least-sum-of-squared-deviations).

  2. k-means cannot be used with arbitrary distances. Technically, it minimizes variance not distance... Study the convergence proof to understand why it does not converge with aebitrary metrics. It requires Bregman divergences.

  3. other algorithms may have other requirements. For example single-link clustering and DBSCAN do not require triangle inequality. DBSCAN does not even require symmetry (and single-link would implicitly use $\min\{ d(x,y), d(y,x)\}$). Other clustering algorithms do not use any distance at all.

  1. Cosine is equivalent to (squared) Euclidean distance on L2 normalized data. So spherical k-means is using a distance. But it's actually using squared Euclidean distance, which does not satisfy the triangle inequality (and matematicians will kill kittens every time you call it a distance). While the nearest center is the same with respect to squared Euclidean and non-squared Euclidean, the optimum center is not. The optimum k-means result is usually not the least-sum-of-distances assignment (but the least-sum-of-squared-deviations).

  2. k-means cannot be used with arbitrary distances. Technically, it minimizes variance not distance... Study the convergence proof to understand why it does not converge with aebitrary metrics. It requires Bregman divergences.

  3. other algorithms may have other requirements. For example single-link clustering and DBSCAN do not require triangle inequality. DBSCAN does not even require symmetry (and single-link would implicitly use $\min\{ d(x,y), d(y,x)\}$). Other clustering algorithms do not use any distance at all. So your question title is bad. Your question is solely about k-means, and even an incorrect interpretation of k-means as minimizing distance.

Source Link

  1. Cosine is equivalent to (squared) Euclidean distance on L2 normalized data. So spherical k-means is using a distance. But it's actually using squared Euclidean distance, which does not satisfy the triangle inequality (and matematicians will kill kittens every time you call it a distance). While the nearest center is the same with respect to squared Euclidean and non-squared Euclidean, the optimum center is not. The optimum k-means result is usually not the least-sum-of-distances assignment (but the least-sum-of-squared-deviations).

  2. k-means cannot be used with arbitrary distances. Technically, it minimizes variance not distance... Study the convergence proof to understand why it does not converge with aebitrary metrics. It requires Bregman divergences.

  3. other algorithms may have other requirements. For example single-link clustering and DBSCAN do not require triangle inequality. DBSCAN does not even require symmetry (and single-link would implicitly use $\min\{ d(x,y), d(y,x)\}$). Other clustering algorithms do not use any distance at all.