2
$\begingroup$

I have to classify some data without any futher prediction (I just need the best clusters on the data). Do I still have to train-test-split my data or do a kfoldCV?

And how do I evaluate my clustering algorithmes (kmeans, spectral, KNN, SOM etc..)? (I already looked the score function of my algorithmes, the KL, the silhouette, inter-intra classes distances)

I think, doing statistic tests would be meaningless, because I don't split the data.

Can you answer those questions?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Well, since you don't know a priori what the clusters would be, having a test set is not only unnecesary, but completely useless! There would be no "labels" on the test set! Just go ahead with your clustering and find something that does the job $\endgroup$
    – David
    Commented Jun 4, 2019 at 9:09
  • $\begingroup$ Thank you for your answer. I tried some algorithmes but the patterns are too different and i can't say what is the best clustering just by looking at the plots. and the KL isn't decreasing. $\endgroup$
    – R_clustering
    Commented Jun 4, 2019 at 11:38

1 Answer 1

Reset to default
1
$\begingroup$

Since you don't have available any specified classes (labels) then you can only use several Evaluation Metrics as Silhouette Coefficient, Rand Index, intra cluster distance etc. depending on which clustering algorithm you are going to use. If you choose k-means or k-medoids then you can also try performing clustering using different values of k considering the Evaluation Metrics mentioned before you can find the appropriate value of k.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for answering. Those evaluation methods are heuristic, that's why I can't be sure of the result (but I still used most of them). About the clustering method, I use SOM (self-organizing map), with 2 dimensions (as far as I know, there is no convergence theorem with the 2d (there exists in 1d), and I compare the SOM with k-means, spectral, and other methods. Concerning the number of cluster, I already concidered different values of the number of clusters. It's kinda weird but I need to concider at least 100 clusters, but SOM and kmeans perfomed better for less. $\endgroup$
    – R_clustering
    Commented Jun 7, 2019 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.