How can K-Means clustering work without spatial information?

Question

Just got stuck at working with K-means clustering.

I have looked up this python/skimage commands:

image_array = image.reshape([-1,3]).astype(np.float32)
kmeans = KMeans(n_clusters=2, random_state=0).fit(image_array)
labels_array = kmeans.labels_
labels = labels_array.reshape([image.shape[0], image.shape[1]])

when I noticed that the RGB images hast to be converted to one long array. How can K-means clustering know about the 2 spatial dimensions (and the 3rd one - color) when I pass an array?

Or is just my assumption wrong that spatial information is needed? At least the goal is to minimize the within-cluster sum of squares. The distance in x, y and color direction is therefor important, isn't it?

Answer 1

The code sample you show actually only clusters pixels according to the color information—the spatial information is not being used. You could add it to the model by appending the actual coordinates to the color dimension:

coords_x, coords_y = np.meshgrid(range(img.shape[0]), range(img.shape[1]))
image_array = image.reshape([-1,3]).astype(np.float32)
image_array = np.concatenate([image_array,
                              coords_x.reshape(-1,1),
                              coords_y.reshape(-1,1)], axis=-1)    
kmeans = KMeans(n_clusters=2, random_state=0).fit(image_array)
labels_array = kmeans.labels_
labels = labels_array.reshape([image.shape[0], image.shape[1]])

However, then you run into trouble of how to weight/scale the individual dimensions, because the distance in RGB space and distance in pixel coordinates are not intuitively comparable. See for example Standardizing some features in K-Means for details on that problem.