3
$\begingroup$

In some clustering algorithm, ex: K-Means cluster, it is very sensitive with outliers, so we need to remove outliers before aplly K-Means, or it will be bad clustering. So :

  1. How can we know some points are outliers if we can not plot it ( high dimension data ) ?
  2. how can we know its K-Means model is good or bad ? Because it is unsupervised learning, so we can not calculate accuracy rate ( something likes F1 score ,... ). Or do we have any method to know an unsupervised learning model is good or bad ?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The answer is checking the loss (sum of distance in k-means setting) on testing data set.

In unsupervised learning, we also need both training and testing data set. Because it is also common to have over-fitting in unsupervised setting. For example, in K-means if we increase number of clusters $k$, the clustering performance (loss function / sum of distance) will always get better in the training data, and eventually will over-fit the data. The extreme example, would be make every data point as a cluster center, then loss is $0$.

To evaluate how good is the clustering algorithm, one valid approach is testing if the model (number of clusters and centers) is still good on a held out testing data.

I was talking about unsupervised learning in general (including probablistic model, mixture of Gaussian etc.), for K-means specifically, you can find answer here (if not duplicate).

How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results?

Improve this answer
$\endgroup$
6
  • $\begingroup$ Why won't increasing k lower the loss value on a test set? $\endgroup$
    – eric_kernfeld
    Sep 27, 2017 at 13:53
  • 1
    $\begingroup$ @eric_kernfeld think about the extreme example, when we put a cluster center in every data point in training. The testing data should have different data points. In such a case, training loss is 0, but testing is definitely not, unless two data sets identical. $\endgroup$
    – Haitao Du
    Sep 27, 2017 at 13:58
  • 2
    $\begingroup$ What you say is true, but not exactly what I am getting at. Suppose you have training and test sets of size 200 and you increase the number of centers from 8 to 16. Don't you think the test set loss will decrease? $\endgroup$
    – eric_kernfeld
    Sep 27, 2017 at 14:10
  • $\begingroup$ Also, how do you use this tactic to compare methods? Two methods may have completely different loss functions, or the loss functions may be arbitrarily scaled. $\endgroup$
    – eric_kernfeld
    Sep 27, 2017 at 14:11
  • $\begingroup$ @eric_kernfeld comparing between different models is another question and not what OP original asked. $\endgroup$
    – Haitao Du
    Sep 27, 2017 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.