1
$\begingroup$

I need to cluster a simple univariate data set into a preset number of clusters. Technically it would closer to binning or sorting the data since it is only 1D, but my boss is calling it clustering, so I'm going to stick to that name. The current method used by the system I'm on is K-means, but that seems like overkill.

Is there a better way of performing this task?

Answers to some other posts are mentioning KDE (Kernel Density Estimation), but that is a density estimation method, how would that work?

I see how KDE returns a density, but how do I tell it to split the data into bins?

How do I have a fixed number of bins independent of the data (that's one of my requirements) ?

More specifically, how would one pull this off using scikit learn?

My input file looks like:

 str ID     sls
 1           10
 2           11 
 3            9
 4           23
 5           21
 6           11  
 7           45
 8           20
 9           11
 10          12

I want to group the sls number into clusters or bins, such that:

Cluster 1: [10 11 9 11 11 12] 
Cluster 2: [23 21 20] 
Cluster 3: [45] 

And my output file will look like:

 str ID     sls    Cluster ID  Cluster centroid
    1        10       1               10.66
    2        11       1               10.66
    3         9       1               10.66 
    4        23       2               21.33   
    5        21       2               21.33
    6        11       1               10.66
    7        45       3               45
    8        20       2               21.33
    9        11       1               10.66 
    10       12       1               10.66
$\endgroup$
  • $\begingroup$ Possible duplicate: stats.stackexchange.com/questions/13781/clustering-1d-data $\endgroup$ – CatsLoveJazz Jan 28 '16 at 23:21
  • 1
    $\begingroup$ @CatsLoveJazz I don't think this is a duplicate: I am asking specifically about KDE. $\endgroup$ – Reinstate Monica Jan 28 '16 at 23:47
  • $\begingroup$ what is the underlying distribution? What kind of sample density are you going to have? What are you going to use this for? $\endgroup$ – EngrStudent - Reinstate Monica Jan 29 '16 at 0:00
  • $\begingroup$ @EngrStudent I don't know what the underlying distribution is, this is real world sales data which I want to group into performance bins. We have sales data for different products p1: 50 Units sold, p2: 30 Units sold, ... p1345: 12 Units sold, there's no time dependency or anything. I'm using about 30 000 data points or so. $\endgroup$ – Reinstate Monica Jan 29 '16 at 0:37
  • $\begingroup$ What package are you using? That is significant because the software limits the methods available. Given the small sample size (less than a few million rows, one column), I would want to use a JMP constellation plot, and the cubic clustering criterion. It doesn't care about 1d vs. 2d. $\endgroup$ – EngrStudent - Reinstate Monica Jan 29 '16 at 0:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.