All Questions
Tagged with clustering k-means
744 questions
4
votes
2
answers
8k
views
Missing data in k-means cluster model
I'm working on clustering email addresses using K-means based on their value to and engagement with the company (metrics such as % of emails opened, # of web browsing sessions, etc). I would like to ...
CommunityBot
- 1
2
votes
0
answers
16
views
How to cluster based on x and y coordinates
I am trying to identify rows in groups of points using clustering algorithms. The bigger picture problem I'm trying to solve is to identify shelves given x and y coordinates of products. I can cluster ...
- 121
4
votes
1
answer
4k
views
How to determine the best batch-size value for Mini Batch K-means algorithm?
I am working on a project where I apply k-means on severals datasets. These datasets may include up to several billion points. I would like to use mini batch k-means to save time. However, the mini ...
CommunityBot
- 1
0
votes
0
answers
11
views
Identify predictors for clustering output?
I have a dataset with variables collected years ago, and many variables collected this year as outcome variables. I want to combine all the variables collected this year to get one outcome, e.g. ...
- 43
5
votes
2
answers
2k
views
Compare clustering results with different attributes and number of clusters
I used K-means to cluster a large data set that has millions of samples. I tried to create the clusters with different sets of attributes, which, as a result, generated different optimal number of ...
CommunityBot
- 1
1
vote
2
answers
384
views
Interpreting results of K-means after PCA
I have this dataset about an airline company customers with 22 explanatory variables. My goal is to perform some sort of customer segmentation with the k-means algorithm. One problem that I've found ...
CommunityBot
- 1
1
vote
0
answers
19
views
Question about running k means cluster analysis
In a previous analysis I had 3 groups of subjects - group x with 35 subjects, control group y with 25 subjects, and control group z with 25 subjects. For each group I have levels of 6 different ...
0
votes
1
answer
237
views
Need help choosing appropriate algorithm for building a Lookalike audience
Newly practicing data scientist here! I am currently stumped on a project and reaching out for some guidance:
I am working with the marketing team in our customer database. There is a small subset of ...
CommunityBot
- 1
25
votes
6
answers
28k
views
How I can convert distance (Euclidean) to similarity score
I am using $k$ means clustering to cluster speaker voices. When I compare an utterance with clustered speaker data I get (Euclidean distance-based) average distortion. This distance can be in range of ...
- 1,783
2
votes
2
answers
485
views
How to include percentage variables in PCA + K-means when some values are undefined because the denominator is 0?
I'm trying to do customer segmentation by using PCA to reduce dimensionality and then feeding the resulting principal components into a K-means algo to get at the final segments. Some of my variables ...
CommunityBot
- 1
2
votes
2
answers
881
views
Jenks Natural breaks - Interpreting Goodness of Variance Fit
I am trying to find breaks in a multiple continuous type variables.
So, I tried the jenks natural breaks algorithm.
Based on the code from here, I managed to find ...
- 59.5k
1
vote
2
answers
486
views
How to find the driving individuals of different k means clusters
My data of 4000 gene expression values across 159 different cells is formatted as so:
...
CommunityBot
- 1
1
vote
1
answer
358
views
Should I scale when clustering text data using K-means?
I want to cluster a folder of texts. I created a data file where for each text, I write whether a certain word appears in it or not. I want to cluster according to this.
So my matrix is globally only ...
CommunityBot
- 1
1
vote
0
answers
40
views
Question on using the elbow method for calculating ideal number of clusters for k means cluster analysis
Newb to cluster analysis here. I have a group of 35 subjects. For all of the subjects I have data for different measures of IQ (verbal, math, etc) and different biomarkers. There are 6 IQ measures in ...
2
votes
2
answers
898
views
How to choose a fair gamma value when performing k-prototypes clustering?
In the k-prototypes clustering algorithm, the distance function consists of two dissimilarity components - one for the numerical elements of the observations, and one for their categorical elements. ...
- 1
1
vote
1
answer
1k
views
How to inform the space and time complexity of K-means, SOM and Hierachical clustering
In the paper I am writing, one of the reviewers asked for an
"a simple computational complexity analysis or time computational demands of their method"
My question is : Can I simply report the ...
CommunityBot
- 1
1
vote
1
answer
489
views
Evaluation of clustering: single cluster solution vs. multiple clusters
There are a few indices out there that help compare competing clustering solutions (e.g., Calinski-Harabasz index and many others).
Is there a popular index/procedure that helps compare a single ...
- 82.8k
0
votes
0
answers
19
views
Clustering Mixed Data Types: Algorithm Selection, Distance Measurement, and Feature Weighting
I have a database of 74,000 records with 29 features. Fourteen of these features are categorical and are either 0 or 1, while the other 15 features are continuous and have been normalized and scaled ...
0
votes
1
answer
244
views
How to interpret contrasting information from the Variation of Information, Dunn and Rand Index for comparing clusterings
There are related questions but the answers don't seem to explain how to practically judge these measurements for non stats users.
I have a dataset which I clustered with K=4 using hierarchical ...
CommunityBot
- 1
2
votes
1
answer
777
views
Clustering spatial data based on location and values
I'm looking for a way, preferably in R, to create a cluster of point data (specifically, the centroids of UK postcodes), where each cluster comes as close as possible to containing a certain number of ...
CommunityBot
- 1
1
vote
1
answer
346
views
Clustering Data with Time and ~10 million records
I have a dataset with features like product categories, their dimensions, price, units sold on a given day.
I want to create clusters out of this dataset (~12-15 million records) and I am using data ...
CommunityBot
- 1
0
votes
0
answers
14
views
calculation of the C-index clustering for manual [duplicate]
Can anyone give me an example of working on the C-index clustering validity test, but calculating manually??
- 1
1
vote
1
answer
411
views
What is the best k in kmeans clustering [duplicate]
I did clustering on a dataset of real-world patients and since the best way to choose the amount of clusters in KMeans clustering is Elbow method and the Silhouette method, I conducted those two and ...
CommunityBot
- 1
1
vote
0
answers
22
views
Spatial Temporal Clustering evenly spaced over time
I have a large dataset of spatio-temporal data. It has longitude and latitude coordinates, and a date for each observation. For example:
Long
Lat
Date
50
20.43
9-19-2010
51
19.5
10-4-2010
51
19.3
...
- 121
0
votes
0
answers
11
views
What are the right metrics to validate the performance of a custom clustering model with three possible outcomes?
I have developed a custom clustering model on top of MiniBatchKmeans, that has three possible outcomes for each data point:
Assign the point to the correct cluster.
Assign the point to the wrong ...
1
vote
1
answer
59
views
Elbow method not giving a proper curve in python code
I am trying to determine how many clusters to use for my k-means clustering using different methods.
first i used the following code to calculate different metrics per cluster number and different ...
- 21
0
votes
0
answers
24
views
Curse of dimensionality in Time series with K-means
I have been looking at the following notebook: time series clustering
where the writer says that the dataset is affected by the "Curse of Dimensionality", so applying TimeSeriesKMeans ...
- 1
3
votes
1
answer
28
views
What is "clall" in index.Gap in "clusterSim" R package?
I am using the "clusterSim" package in my project (https://cran.r-project.org/web/packages/clusterSim/clusterSim.pdf, page 39) and I do not understand the meaning of the "clall" ...
- 94.1k
0
votes
0
answers
16
views
Should the same environmental variable measured with different methods be removed before K-means? What about variables repr. sep. and by their ratio?
So I'm running K-means clustering algorithm on environmental variables measured on different locations. The aim is to see if the environmental variables can be clustered into separate clusters.
Same ...
- 77
1
vote
0
answers
125
views
k-means clustering on a probability distribution instead of a dataset
Normally, clustering algorithms such as $k$-means are defined on a dataset in the following sense: if $D$ is a dataset, find a partition of $D$ into sets $\{S_1, \dots, S_n\}$ that minimises the ...
- 82.8k
2
votes
1
answer
204
views
What is the standard threshold value that is best for accuracy when employing Euclidean distance as a metric for gauging textual similarity?
I'm using Euclidean distance as a metric to compare two sentences for similarity while clustering them using my custom incremental KMeans algorithm. The current threshold value I'm using is 0.7 which ...
- 46k
160
votes
7
answers
134k
views
Clustering on the output of t-SNE
I've got an application where it'd be handy to cluster a noisy dataset before looking for subgroup effects within the clusters. I first looked at PCA, but it takes ~30 components to get to 90% of the ...
- 3,398
0
votes
0
answers
10
views
What is normalized winning frequency in kernel self organizing map(SOM)?
In the k-means based kernel SOM, proposed by MacDonald and Fyfe (2000), the update of the mean is based on a soft learning algorithm
mi(t + 1) = mi(t) + Λ[φ(x) − mi(t)]
where Λ is the normalized ...
0
votes
0
answers
43
views
Why does this K-Means cluster example show 'overlap' between clusters?
I was reading the hypertools docs and came across this pictorial that shows 10 clusters (some seem to share very similar coloring) generated from some (mushroom) ...
- 12.1k
1
vote
0
answers
68
views
K-means clustering - weird PCA visualization
I performed PCA on 4 variables and are shown in this visualization:
At first look it doesn't look convincing and the some clusters seem weird.
The data was cleaned and standardized beforehand. Only ...
- 11
0
votes
0
answers
21
views
Method for pairwise ordering two datasets
Given two rather small but unordered multidimensional vectors/datasets (e.g sets of a handful of 3D coordinates), what is a simple method for pairwise alignment/ordering?
I've though about using ...
- 161
4
votes
3
answers
17k
views
How to split data into training and testing for clustering
I want to use k-means clustering on my dataset to capture the similarity based on two attributes for two groups. I am looking to split my data first into training and testing, and then find clusters ...
2
votes
1
answer
706
views
For which clustering algorithms is the Gap statistic useful?
How can i know for which clustering algorithms (with a parameter that represents number of clusters) it makes sense to use the Gap statistic? I've read in the paper by Tibshirani, Walter & Hastie ...
CommunityBot
- 1
0
votes
1
answer
53
views
Continuous monitoring of KMeans model post production
In the process of deploying a KMeans model for a customer segmentation use case into production. KMeans doesn’t produce the same results every time and after production cluster sizes and arrangements ...
- 870
72
votes
6
answers
157k
views
Is it important to scale data before clustering?
I found this tutorial, which suggests that you should run the scale function on features before clustering (I believe that it converts data to z-scores).
I'm wondering whether that is necessary. I'm ...
- 5,541
5
votes
1
answer
275
views
When should I use classical k-means clustering and when should I use trimmed k-means clustering?
I suspect that if there are many unimportant outliers, trimmed k-mean clustering should be employed.
Am I on the right track?
- 2,939
4
votes
2
answers
409
views
Question about Silhouette index calculation using scikit
I am currently working with continuous data measured from different sensors (thermometers and voltmeters). I have a matrix whose columns represent the sensors and the rows are normalized measurements (...
- 28.3k
1
vote
1
answer
355
views
K-means dominated by one or two variables only
What should we do if clustering such as K-means is dominated by one or two variables in the list of used variables? Shall we leave the other variables?
CommunityBot
- 1
0
votes
0
answers
41
views
Turning heatmap into clusters - Classification
Assume that you having a heatmap that looks like this. The goal is to classify all the "dot" inside the image. How can that be done?
The assumptions of the image:
The image has always black ...
- 425
1
vote
1
answer
110
views
How to tell whether segments from K Means clustering result are "successful" and will impact business metrics?
Background
I'm a data analyst. The Business unit I'm assigned for needs to segment users based on power vs non-power users so they can target each segment with proper treatments.
Goal
Segment users (...
- 141k
0
votes
1
answer
337
views
Dummy Variable Trap in KMeans Clustering
My data set is having a column Gender, so I have to apply One Hot Encodingto perform KMeans Clustering.
Q1. Should I take care about ...
- 141k
0
votes
2
answers
5k
views
Clustering Data Using Gower and Kmeans
I am trying to do clustering on my data which consists of both categorical and continuous variables. I have some questions which I would like to ask:
I am going to use the Gower Distance measure to ...
- 153
2
votes
1
answer
1k
views
Clustering leading to visually overlapping clusters on scatterplot
I am dealing with a dataset with 13 features.
After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters.
Now the point is that, although the ...
CommunityBot
- 1
2
votes
1
answer
160
views
Clustering algorithms puts data points that are visually far apart in same cluster
I am trying to cluster a very large set of data points, of roughly (20000, 100) shape. I could not run density based DBSCAN or SpectralClustering due to the ...
- 29
1
vote
1
answer
265
views
Clustering Analysis for large data in R
I am trying to perform a clustering analysis for a csv file with 50k+ rows, 10 columns. I tried k-mean, hierarchical and model based clustering methods. Only k-mean works because of the large data set....
- 157