I have converted all of my features to binary variables. now I have 21 features in my data set. I am trying to cluster them with k-means. I used Hamming distance in order to measure the distance between every instance and centroids at each steps.

But when I was trying to calculate the mean (in order to have a new centroid), I realized that taking a mean of binary variables does not make sense.

After doing some research I decided to use mode instead of mean. I used modes like this:

enter image description here

the rest of the algorithm is the same as k-means. but the problem is my error rate is too high.

my question is: Am I doing it correctly? Do you have any suggestion for me to deal with these data? [if I only want the k-means output for this dataset]

update 1

I tried medians instead of modes (the same approach) and the result is still suffering from high error rate.

update 2

before clustering, I know about 70 percent of instances in my data set belongs to one group and about 25 percent belongs to another group. I think they affect the result of clustering. am I right?

  • 1
    $\begingroup$ 1) Do not do k-means with binary data, it is both theoretically questionable and crude. Do, for example, hierarchical clustering or k-medoids with a suitable for you distance function for binary data. 2) What is your "modified" k-means you did - you haven't explained. Note that k-means algorithm can converge to a nice optimum only when it minimizes euclidean deviations from centroid (not mode). $\endgroup$ – ttnphns Jan 21 '18 at 20:11
  • $\begingroup$ I just coded the algorithm from scratch and didn't change the k-means algorithm except the part for calculating the means. I used modes and medians. both useless... $\endgroup$ – Adel Jan 21 '18 at 20:57
  • 2
    $\begingroup$ Means work just fine for binary variables. For instance, about 51% of babies are female: that's a mean. It provides far more information than reporting that the mode and median are both female. $\endgroup$ – whuber Jan 21 '18 at 22:54
  • $\begingroup$ @whuber for binary data, kmodes is supposedly better than kmeans. I guess because the cluster centers then are possible data points, whereas with the mean they are no longer binary. Maybe also the L2 objective of kmeans isn't that meaningful for binary data (but yes, you can give a variance for the gender variable...) $\endgroup$ – Has QUIT--Anony-Mousse Jan 26 '18 at 20:02

I'd rather consider frequent itemset mining.

I think the problems you see arise from two assumptions:

  1. each object belongs to exactly one cluster
  2. each attribute has the same importance

Also, how do you evaluate? What is the "error rate" you are using?

  • $\begingroup$ I have the actual clusters. I mean the Cluster related to each instance. I used clustering without this info. After clustering, I go through each instance and check it's newly assigned cluster label and compare this with the real/actual cluster label. The errors are high. $\endgroup$ – Adel Jan 24 '18 at 20:37
  • $\begingroup$ Why do you need to cluster then, if you already know the answer? Also, it is naive to assume that cluster labels = class labels. $\endgroup$ – Has QUIT--Anony-Mousse Jan 25 '18 at 18:53
  • $\begingroup$ To build a model to cluster new instances. for evaluating a clustering algorithm you need to know error rate don't you? In addition, I didn't consider the assumption you mentioned! $\endgroup$ – Adel Jan 26 '18 at 11:45
  • $\begingroup$ You classify new instances, but you don't cluster them. $\endgroup$ – Has QUIT--Anony-Mousse Jan 26 '18 at 20:01
  • $\begingroup$ I didn't say classification. we do cluster new instances. online clustering algorithms as an example is used to learn continuously evolving clusters. $\endgroup$ – Adel Jan 27 '18 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.