All Questions
Tagged with clustering k-means
744 questions
445
votes
5
answers
177k
views
How to understand the drawbacks of K-means
K-means is a widely used method in cluster analysis. In my understanding, this method does NOT require ANY assumptions, i.e., give me a dataset and a pre-specified number of clusters, k, and I just ...
- 6,919
160
votes
7
answers
134k
views
Clustering on the output of t-SNE
I've got an application where it'd be handy to cluster a noisy dataset before looking for subgroup effects within the clusters. I first looked at PCA, but it takes ~30 components to get to 90% of the ...
- 13.7k
118
votes
6
answers
176k
views
What is the relation between k-means clustering and PCA?
It is a common practice to apply PCA (principal component analysis) before a clustering algorithm (such as k-means). It is believed that it improves the clustering results in practice (noise reduction)...
- 4,458
97
votes
6
answers
172k
views
Why does k-means clustering algorithm use only Euclidean distance metric?
Is there a specific purpose in terms of efficiency or functionality why the k-means algorithm does not use for example cosine (dis)similarity as a distance metric, but can only use the Euclidean norm? ...
- 1,111
92
votes
6
answers
83k
views
How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results?
How would you know if your (high dimensional) data exhibits enough clustering so that results from kmeans or other clustering algorithm is actually meaningful?
For k-means algorithm in particular, ...
- 2,218
72
votes
6
answers
157k
views
Is it important to scale data before clustering?
I found this tutorial, which suggests that you should run the scale function on features before clustering (I believe that it converts data to z-scores).
I'm wondering whether that is necessary. I'm ...
- 1,479
69
votes
2
answers
100k
views
Are mean normalization and feature scaling needed for k-means clustering?
What are the best (recommended) pre-processing steps before performing k-means?
- 1,373
64
votes
3
answers
68k
views
Clustering with K-Means and EM: how are they related?
I have studied algorithms for clustering data (unsupervised learning): EM, and k-means.
I keep reading the following :
k-means is a variant of EM, with the assumptions that clusters are
...
- 793
59
votes
11
answers
99k
views
How to decide on the correct number of clusters?
We find the cluster centers and assign points to k different cluster bins in k-means clustering which is a very well known algorithm and is found almost in every machine learning package on the net. ...
- 1,725
58
votes
3
answers
116k
views
Clustering a long list of strings (words) into similarity groups
I have the following problem at hand: I have a very long list of words, possibly names, surnames, etc. I need to cluster this word list, such that similar words, for example words with similar edit (...
- 2,310
40
votes
2
answers
41k
views
How to use both binary and continuous variables together in clustering?
I need to use binary variables (values 0 & 1) in k-means. But k-means only works with continuous variables. I know some people still use these binary variables in k-means ignoring the fact that k-...
- 5,683
40
votes
5
answers
64k
views
Clustering a dataset with both discrete and continuous variables
I have a dataset X which has 10 dimensions, 4 of which are discrete values.
In fact, those 4 discrete variables are ordinal, i.e. a higher value implies a higher/better semantic.
2 of these discrete ...
- 611
39
votes
2
answers
57k
views
How would PCA help with a k-means clustering analysis?
Background: I want to classify the residential areas of a city into groups based on their social-economic characteristics, including housing unit density, population density, green space area, housing ...
- 605
38
votes
4
answers
84k
views
Determine different clusters of 1d data from database
I have a database table of data transfers between different nodes. This is a huge database (with nearly 40 million transfers). One of the attributes is the number of bytes (nbytes) transfers which ...
- 381
35
votes
4
answers
50k
views
Clustering a correlation matrix
I have a correlation matrix which states how every item is correlated to the other item. Hence for a N items, I already have a N*N correlation matrix. Using this correlation matrix how do I cluster ...
- 483
35
votes
1
answer
35k
views
Difference between standard and spherical k-means algorithms
I would like to understand, what is the major implementation difference between standard and spherical k-means clustering algorithms.
In each step, k-means computes distances between element vectors ...
- 1,329
33
votes
5
answers
26k
views
Clustering procedure where each cluster has an equal number of points?
I have some points $X=\{x_1,...,x_n\}$ in $R^p$, and I want to cluster the points so that:
Each cluster contains an equal number of elements of $X$. (Assume that the number of clusters divides $n$.)
...
- 381
31
votes
8
answers
36k
views
Perform K-means (or its close kin) clustering with only a distance matrix, not points-by-features data
I want to perform K-means clustering on objects I have, but the objects aren't described as points in space, i.e. by objects x features dataset. However, I am able ...
- 313
28
votes
2
answers
24k
views
How do I know my k-means clustering algorithm is suffering from the curse of dimensionality?
I believe that the title of this question says it all.
- 383
27
votes
4
answers
60k
views
Estimating the most important features in a k-means cluster partition
Is there a way to determine which features / variables of the dataset are the most important / dominant within a k-means cluster solution?
- 373
26
votes
3
answers
39k
views
Visually plotting multi dimensional cluster data
I have a data set with 16 variables, and after clustering by kmeans, I wish to plot the two groups.
What plots do you suggest to visually represent the two clusters?
- 873
26
votes
2
answers
26k
views
If k-means clustering is a form of Gaussian mixture modeling, can it be used when the data are not normal?
I'm reading Bishop on EM algorithm for GMM and the relationship between GMM and k-means.
In this book it says that k-means is a hard assign version of GMM. I'm wondering does that imply that if the ...
- 527
25
votes
6
answers
28k
views
How I can convert distance (Euclidean) to similarity score
I am using $k$ means clustering to cluster speaker voices. When I compare an utterance with clustered speaker data I get (Euclidean distance-based) average distortion. This distance can be in range of ...
- 381
24
votes
6
answers
48k
views
Why doesn't k-means give the global minimum?
I read that the k-means algorithm only converges to a local minimum and not to a global minimum. Why is this? I can logically think of how initialization could affect the final clustering and there is ...
- 589
22
votes
3
answers
27k
views
Why does gap statistic for k-means suggest one cluster, even though there are obviously two of them?
I am using K-means to cluster my data and was looking for a way to suggest an "optimal" cluster number. Gap statistics seems to be a common way to find a good cluster number.
For some reason it ...
- 1,239
22
votes
4
answers
39k
views
k-means implementation with custom distance matrix in input
Can anyone point me out a k-means implementation (it would be better if in matlab) that can take the distance matrix in input?
The standard matlab implementation needs the observation matrix in input ...
- 341
22
votes
3
answers
38k
views
Do I need to drop variables that are correlated/collinear before running kmeans?
I am running kmeans to identify clusters of customers. I have approximately 100 variables to identify clusters. Each of these variables represent the % of spend by a customer on a category. So, if I ...
- 221
21
votes
4
answers
25k
views
How to understand the drawbacks of Hierarchical Clustering?
Can someone explain the pros and cons of Hierarchical Clustering?
Does Hierarchical Clustering have the same drawbacks as K means?
What are the advantages of Hierarchical Clustering over K means?
...
- 5,683
19
votes
4
answers
25k
views
In cluster analysis, how does Gaussian mixture model differ from K Means when we know the clusters are spherical?
I understand how main difference between K-mean and Gaussian mixture model (GMM) is that K-Mean only detects spherical clusters and GMM can adjust its self to elliptic shape cluster. However, how do ...
- 313
19
votes
2
answers
22k
views
Clustering of very skewed, count data: any suggestions to go about (transform etc)?
Basic problem
Here is my basic problem: I am trying to cluster a dataset containing some very skewed variables with counts. The variables contain many zeros and are therefore not very informative for ...
- 291
19
votes
4
answers
29k
views
What do you do when there's no elbow point for kmeans clustering
I've learned that when choosing a number of clusters, you should look for an elbow point for different values of K. I've plotted the values of withinss for values of k from 1 to 10, but I'm not seeing ...
- 1,479
18
votes
4
answers
13k
views
Are there any non-distance based clustering algorithms?
It seems that for K-means and other related algorithms, clustering is based off calculating distance between points. Is there one that works without it?
- 578
17
votes
5
answers
22k
views
What algorithm should I use to cluster a huge binary dataset into few categories?
I have a large (650K rows * 62 columns) matrix of binary data (0-1 entries only). The matrix is mostly sparse: about 8% is filled.
I would like to cluster it into 5 groups - say named from 1 to 5. I ...
- 171
16
votes
2
answers
36k
views
k-means vs k-median?
I know there is k-means clustering algorithm and k-median. One that uses the mean as the center of the cluster and the other uses the median. My question is: when/where to use which?
- 8,411
16
votes
2
answers
19k
views
Is there a function in R that takes the centers of clusters that were found and assigns clusters to a new data set
I have two parts of a multidimensional data set, let's call them train and test. And I want to built a model based on the train ...
- 329
14
votes
3
answers
42k
views
Why do we use k-means instead of other algorithms?
I researched about k-means and these are what I got: k-means is one of the simplest algorithm which uses unsupervised learning method to solve known clustering issues. It works really well with large ...
14
votes
3
answers
18k
views
Clustering probability distributions - methods & metrics?
I have some data points, each containing 5 vectors of agglomerated discrete results, each vector's results generated by a different distribution, (the specific kind of which I am not sure, my best ...
- 301
13
votes
2
answers
50k
views
Interpreting result of k-means clustering in R
I was using the kmeans instruction of R for performing the k-means algorithm on Anderson's iris dataset. I have a question about some parameters that I got. The ...
- 251
13
votes
4
answers
28k
views
Initializing K-means centers by the means of random subsamples of the dataset?
If I have a certain dataset, how smart would it be to initialize cluster centers using means of random samples of that dataset?
For example, suppose I want ...
- 3,875
13
votes
4
answers
13k
views
Methods of initializing K-means clustering
I am interested in the current state of the art for selecting initial seeds (cluster centers) for K-means.
Googling leads to two popular choices:
random selection of initial seeds, and,
using the ...
- 303
13
votes
2
answers
14k
views
Assumption of equal size of clusters in clustering
I am wondering: when clustering data using some general algorithm is there is an assumption on approximately equal sizes of the clusters?
For example, in k-means as I know all clusters should have ...
- 911
13
votes
1
answer
8k
views
k-means|| a.k.a. Scalable K-Means++
Bahman Bahmani et al. introduced k-means||, which is a faster version of k-means++.
This algorithm is taken from page 4 of their paper, Bahmani, B., Moseley, B., Vattani, A., Kumar, R., & ...
- 369
12
votes
3
answers
13k
views
Clustering as dimensionality reduction
I'm reading a book "Machine learning with Spark" by Nick Pentreath, and at page 224-225 the author discusses about using K-means as a form of dimensionality reduction.
I have never seen this kind of ...
- 1,260
12
votes
2
answers
24k
views
Compute BIC clustering criterion (to validate clusters after K-means)
I'm wondering if there is a good way to calculate the clustering criterion based on BIC formula, for a k-means output in R? I'm a bit confused as to how to calculate that BIC so that I can compare it ...
- 251
12
votes
2
answers
9k
views
Does k-means have any advantages over HDBSCAN expect for runtime?
I have recently learned about HDBSCAN (a fairly new method for clustering, not yet available in scikit-learn) and am really surprised at how good it is. The following picture illustrates that the ...
- 263
12
votes
1
answer
32k
views
Clustering inertia formula in scikit learn
I would like to code a kmeans clustering in python using pandas and scikit learn. In order to select the good k, I would like to code the Gap Statistic from Tibshirani and al 2001 (pdf).
I would like ...
- 812
11
votes
1
answer
36k
views
usefulness of k-means clustering on high dimensional data [duplicate]
I wonder what is the usefulness of k-means clustering in high dimensional spaces, and why it can be better (or not) than other clustering methods when dealing with high dimensional spaces.
- 213
11
votes
4
answers
3k
views
Are there cases where there is no optimal k in k-means?
This has been inside my mind for at least a few hours. I was trying to find an optimal k for the output from the k-means algorithm (with a cosine similarity metric) so I ended up plotting the ...
- 4,552
11
votes
3
answers
12k
views
Choosing clusters for k-means: the 1 cluster case
Does anyone know a good method to determine if clustering using kmeans is even appropriate? That is, what if your sample is actually homogenous? I know something like a mixture model (via mclust in R) ...
- 3,393
11
votes
2
answers
7k
views
Difference between PCA and spectral clustering for a small sample set of Boolean features
I have a dataset of 50 samples. Each sample is composed of 11 (possibly correlated) Boolean features. I would like to some how visualize these samples on a 2D plot and examine if there are clusters/...
- 271