I clustered some data (rows: text documents, columns: word frequencies) using the KMeans implementation in Scikit Learn. This, like most other centroid-based clustering algorithms, returns a centroid for each cluster.
I am now trying to identify features predictive of each cluster by simply comparing the value for a certain feature in a centroid to the other centroids. I just subtract the value of centroid 1's feature X from the value of centroid 2's feature X (cf. below) and then sort the differences to see which features have the widest spread and and are thus most strongly correlated with a cluster.
- Does this approach make sense / work?
- If not, why not and what should I do instead? (I found this but don't know how exhaustive it is)
I feel that the approach is kind of simplistic, but can't think of a much better way either.
More detailed description of idea, just for clarification:
In a 3 cluster k means, I have the vector for centroid 1 [1,2,0,...]. for centroid 2 [10, 2, 400,...] and centroid 3 [100, 2, 0, ...]. I then subtract feature 1: abs(1-10), abs(1-100), feature 2: abs(2-2), abs(2-2), etc. This I would interpret to mean that feature 1 is strongly predictive of cluster 3 membership, while feature 2 is not very predictive at all. In real life, a feature of course translates into a word. All my features are normalized as feature/total words. Any help is much appreciated!