Suppose we have a data set generated from k different distributions.
In k-means, the data classification step (in which we associate each data point to the nearest centroid) uses the current centroids to generate new data clusters. We can say that the better the centroids positions are, the better the classification is.
My doubts comes when we talk about the relation between centroids and the real data means. We can't say that k-means will always find the real means, but the resulting centroids after a full execution of this method will certainly be close to them.
Considering that the classification step means only associating each point to the closest centroid, can we say that the real means are the average optimal centroid positions for data classification (classifying data only once, with no centroid update)?
I thought the real mean of a distribution is the point that minimizes the sum of squared error when we generate a high amount of data points with that distribution, so I guessed it would be reasonable to say that.
I would appreciate if you guys could point me any references about this point. Thanks in advance!