2
$\begingroup$

I have converted all of my features to binary variables. now I have 21 features in my data set. I am trying to cluster them with k-means. I used Hamming distance in order to measure the distance between every instance and centroids at each steps.

But when I was trying to calculate the mean (in order to have a new centroid), I realized that taking a mean of binary variables does not make sense.

After doing some research I decided to use mode instead of mean. I used modes like this:

enter image description here

the rest of the algorithm is the same as k-means. but the problem is my error rate is too high.

my question is: Am I doing it correctly? Do you have any suggestion for me to deal with these data? [if I only want the k-means output for this dataset]

update 1

I tried medians instead of modes (the same approach) and the result is still suffering from high error rate.

update 2

before clustering, I know about 70 percent of instances in my data set belongs to one group and about 25 percent belongs to another group. I think they affect the result of clustering. am I right?

$\endgroup$
4
  • 1
    $\begingroup$ 1) Do not do k-means with binary data, it is both theoretically questionable and crude. Do, for example, hierarchical clustering or k-medoids with a suitable for you distance function for binary data. 2) What is your "modified" k-means you did - you haven't explained. Note that k-means algorithm can converge to a nice optimum only when it minimizes euclidean deviations from centroid (not mode). $\endgroup$
    – ttnphns
    Jan 21, 2018 at 20:11
  • $\begingroup$ I just coded the algorithm from scratch and didn't change the k-means algorithm except the part for calculating the means. I used modes and medians. both useless... $\endgroup$
    – Adel
    Jan 21, 2018 at 20:57
  • 2
    $\begingroup$ Means work just fine for binary variables. For instance, about 51% of babies are female: that's a mean. It provides far more information than reporting that the mode and median are both female. $\endgroup$
    – whuber
    Jan 21, 2018 at 22:54
  • $\begingroup$ @whuber for binary data, kmodes is supposedly better than kmeans. I guess because the cluster centers then are possible data points, whereas with the mean they are no longer binary. Maybe also the L2 objective of kmeans isn't that meaningful for binary data (but yes, you can give a variance for the gender variable...) $\endgroup$ Jan 26, 2018 at 20:02

1 Answer 1

2
$\begingroup$

I'd rather consider frequent itemset mining.

I think the problems you see arise from two assumptions:

  1. each object belongs to exactly one cluster
  2. each attribute has the same importance

Also, how do you evaluate? What is the "error rate" you are using?

$\endgroup$
10
  • $\begingroup$ I have the actual clusters. I mean the Cluster related to each instance. I used clustering without this info. After clustering, I go through each instance and check it's newly assigned cluster label and compare this with the real/actual cluster label. The errors are high. $\endgroup$
    – Adel
    Jan 24, 2018 at 20:37
  • $\begingroup$ Why do you need to cluster then, if you already know the answer? Also, it is naive to assume that cluster labels = class labels. $\endgroup$ Jan 25, 2018 at 18:53
  • $\begingroup$ To build a model to cluster new instances. for evaluating a clustering algorithm you need to know error rate don't you? In addition, I didn't consider the assumption you mentioned! $\endgroup$
    – Adel
    Jan 26, 2018 at 11:45
  • $\begingroup$ You classify new instances, but you don't cluster them. $\endgroup$ Jan 26, 2018 at 20:01
  • $\begingroup$ I didn't say classification. we do cluster new instances. online clustering algorithms as an example is used to learn continuously evolving clusters. $\endgroup$
    – Adel
    Jan 27, 2018 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.