Determine different clusters of 1d data from database I have a database table of data transfers between different nodes. This is a huge database (with nearly 40 million transfers). One of the attributes is the number of bytes (nbytes) transfers which range from 0 bytes to 2 tera bytes. I would like to cluster the nbytes such that given k clusters some x1 transfers belongs to k1 cluster, x2 transfters to k2 etc. 
From the terminology that I used you might have guessed what I was going with: K-means. This is 1d data since nbytes is the only feature I care about. When I was searching for different methods to this I saw the EM was mentioned a couple times along with a non-clustering approach. I would like to know about your views on how to approach this problem (specifically whether to cluster or not to cluster). 
Thanks!
 A: In one dimensional data, don't use cluster analysis.
Cluster analysis is usually a multivariate technique. Or let me better put it the other way around: for one-dimensional data -- which is completely ordered -- there are much better techniques. Using k-means and similar techniques here is a total waste, unless you put in enough effort to actually optimize them for the 1-d case.
Just to give you an example: for k-means it is common to use k random objects as initial seeds. For one dimensional data, it's fairly easy to do better by just using the appropriate quantiles (1/2k, 3/2k, 5/2k etc.), after sorting the data once, and then optimize from this starting point. However, 2D data cannot be sorted completely. And in a grid, there likely will be empty cells. 
I also wouldn't call it cluster. I would call it interval. What you really want to do is to optimize the interval borders. If you do k-means, it will test for each object if it should be moved to another cluster. That does not make sense in 1D: only the objects at the interval borders need to be checked. That obviously is much faster, as there are only ~2k objects there. If they do not already prefer other intervals, more central objects will not either.
You may want to look into techniques such as Jenks Natural Breaks optimization, for example.
Or you can do a kernel density estimation and look for local minima of the density to split there. The nice thing is that you do not need to specify k for this!
See this answer for an example how to do this in Python (green markers are the cluster modes; red markers a points where the data is cut; the y axis is a log-likelihood of the density):

P.S. please use the search function. Here are some questions on 1-d data clustering that you missed:


*

*Clustering 1D data

*https://stackoverflow.com/questions/7869609/cluster-one-dimensional-data-optimally

*https://stackoverflow.com/questions/11513484/1d-number-array-clustering
A: One-dimensional clustering can be done optimally and efficiently, which may be able to give you insight on the structure of your data.
In the one-dimensional case, there are methods that are optimal and efficient (O(kn)), and as a bonus there are even regularized clustering algorithms that will let you automatically select the number of clusters! I recommend this survey: https://cs.au.dk/~larsen/papers/1dkmeans.pdf
R implementations can be found on the Ckmeans.1d.dp package:
https://cran.r-project.org/web/packages/Ckmeans.1d.dp/index.html
As a side note, 1-dimensional clustering can be used for quantization, where you represent your input data using a smaller set of values; this can help with compression, or to speed up searching for example.
A: Is your question whether you should cluster or what method you should use to cluster?
Regarding whether you should cluster, it depends if you want to automatically partition your data (for example if you want to repeat this partitioning several times). If you are doing this only once, you can just look at the histogram of the distribution of your values, and partition it by eye, as proposed in the comments. I would recommend looking at the data by eye anyway, since it could help you determine how many clusters you want and also whether the clustering "worked".
Regarding the type of clustering, k-means should be fine if there are "real" clusters in the data. If you don't see any clusters in the histogram, it doesn't make much sense clustering it anyway, since any partitioning of your data range will give valid clusters (or in the case of random initiation of kmeans, you will get different clusters each run).
A: You can try:


*

*KMeans, GMM or other methods by specifying n_clusters= no. of peaks in kernel density plot.

*KMeans, GMM or other methods by determining the optimum no. of clusters based on some metrics. More info: [here] https://en.wikipedia.org/wiki/Determining_the_number_of_clusters_in_a_data_set
