All Questions
Tagged with clustering k-means
744 questions
2
votes
0
answers
16
views
How to cluster based on x and y coordinates
I am trying to identify rows in groups of points using clustering algorithms. The bigger picture problem I'm trying to solve is to identify shelves given x and y coordinates of products. I can cluster ...
- 121
0
votes
0
answers
11
views
Identify predictors for clustering output?
I have a dataset with variables collected years ago, and many variables collected this year as outcome variables. I want to combine all the variables collected this year to get one outcome, e.g. ...
- 43
1
vote
0
answers
19
views
Question about running k means cluster analysis
In a previous analysis I had 3 groups of subjects - group x with 35 subjects, control group y with 25 subjects, and control group z with 25 subjects. For each group I have levels of 6 different ...
1
vote
0
answers
40
views
Question on using the elbow method for calculating ideal number of clusters for k means cluster analysis
Newb to cluster analysis here. I have a group of 35 subjects. For all of the subjects I have data for different measures of IQ (verbal, math, etc) and different biomarkers. There are 6 IQ measures in ...
0
votes
0
answers
19
views
Clustering Mixed Data Types: Algorithm Selection, Distance Measurement, and Feature Weighting
I have a database of 74,000 records with 29 features. Fourteen of these features are categorical and are either 0 or 1, while the other 15 features are continuous and have been normalized and scaled ...
0
votes
0
answers
14
views
calculation of the C-index clustering for manual [duplicate]
Can anyone give me an example of working on the C-index clustering validity test, but calculating manually??
- 1
1
vote
0
answers
22
views
Spatial Temporal Clustering evenly spaced over time
I have a large dataset of spatio-temporal data. It has longitude and latitude coordinates, and a date for each observation. For example:
Long
Lat
Date
50
20.43
9-19-2010
51
19.5
10-4-2010
51
19.3
...
- 121
0
votes
0
answers
11
views
What are the right metrics to validate the performance of a custom clustering model with three possible outcomes?
I have developed a custom clustering model on top of MiniBatchKmeans, that has three possible outcomes for each data point:
Assign the point to the correct cluster.
Assign the point to the wrong ...
0
votes
0
answers
24
views
Curse of dimensionality in Time series with K-means
I have been looking at the following notebook: time series clustering
where the writer says that the dataset is affected by the "Curse of Dimensionality", so applying TimeSeriesKMeans ...
- 1
3
votes
1
answer
28
views
What is "clall" in index.Gap in "clusterSim" R package?
I am using the "clusterSim" package in my project (https://cran.r-project.org/web/packages/clusterSim/clusterSim.pdf, page 39) and I do not understand the meaning of the "clall" ...
- 51
0
votes
0
answers
16
views
Should the same environmental variable measured with different methods be removed before K-means? What about variables repr. sep. and by their ratio?
So I'm running K-means clustering algorithm on environmental variables measured on different locations. The aim is to see if the environmental variables can be clustered into separate clusters.
Same ...
- 77
1
vote
0
answers
125
views
k-means clustering on a probability distribution instead of a dataset
Normally, clustering algorithms such as $k$-means are defined on a dataset in the following sense: if $D$ is a dataset, find a partition of $D$ into sets $\{S_1, \dots, S_n\}$ that minimises the ...
- 111
2
votes
1
answer
204
views
What is the standard threshold value that is best for accuracy when employing Euclidean distance as a metric for gauging textual similarity?
I'm using Euclidean distance as a metric to compare two sentences for similarity while clustering them using my custom incremental KMeans algorithm. The current threshold value I'm using is 0.7 which ...
- 21
0
votes
0
answers
10
views
What is normalized winning frequency in kernel self organizing map(SOM)?
In the k-means based kernel SOM, proposed by MacDonald and Fyfe (2000), the update of the mean is based on a soft learning algorithm
mi(t + 1) = mi(t) + Λ[φ(x) − mi(t)]
where Λ is the normalized ...
0
votes
0
answers
43
views
Why does this K-Means cluster example show 'overlap' between clusters?
I was reading the hypertools docs and came across this pictorial that shows 10 clusters (some seem to share very similar coloring) generated from some (mushroom) ...
1
vote
0
answers
68
views
K-means clustering - weird PCA visualization
I performed PCA on 4 variables and are shown in this visualization:
At first look it doesn't look convincing and the some clusters seem weird.
The data was cleaned and standardized beforehand. Only ...
- 11
0
votes
0
answers
21
views
Method for pairwise ordering two datasets
Given two rather small but unordered multidimensional vectors/datasets (e.g sets of a handful of 3D coordinates), what is a simple method for pairwise alignment/ordering?
I've though about using ...
- 161
1
vote
1
answer
59
views
Elbow method not giving a proper curve in python code
I am trying to determine how many clusters to use for my k-means clustering using different methods.
first i used the following code to calculate different metrics per cluster number and different ...
- 11
0
votes
1
answer
53
views
Continuous monitoring of KMeans model post production
In the process of deploying a KMeans model for a customer segmentation use case into production. KMeans doesn’t produce the same results every time and after production cluster sizes and arrangements ...
- 65
4
votes
2
answers
409
views
Question about Silhouette index calculation using scikit
I am currently working with continuous data measured from different sensors (thermometers and voltmeters). I have a matrix whose columns represent the sensors and the rows are normalized measurements (...
- 205
0
votes
0
answers
41
views
Turning heatmap into clusters - Classification
Assume that you having a heatmap that looks like this. The goal is to classify all the "dot" inside the image. How can that be done?
The assumptions of the image:
The image has always black ...
- 425
1
vote
1
answer
110
views
How to tell whether segments from K Means clustering result are "successful" and will impact business metrics?
Background
I'm a data analyst. The Business unit I'm assigned for needs to segment users based on power vs non-power users so they can target each segment with proper treatments.
Goal
Segment users (...
- 135
0
votes
1
answer
337
views
Dummy Variable Trap in KMeans Clustering
My data set is having a column Gender, so I have to apply One Hot Encodingto perform KMeans Clustering.
Q1. Should I take care about ...
2
votes
1
answer
160
views
Clustering algorithms puts data points that are visually far apart in same cluster
I am trying to cluster a very large set of data points, of roughly (20000, 100) shape. I could not run density based DBSCAN or SpectralClustering due to the ...
- 29
1
vote
2
answers
384
views
Interpreting results of K-means after PCA
I have this dataset about an airline company customers with 22 explanatory variables. My goal is to perform some sort of customer segmentation with the k-means algorithm. One problem that I've found ...
1
vote
1
answer
1k
views
Elbow method Vs Gap statistics, which one? challenging for data scientist
I am working on hourly-weather data. It contains four features: rain, wind speed, humidity, and temperature. Obviously, all of them are continuous values. The number of records is around 17000. Other ...
- 73
0
votes
0
answers
58
views
How to identify the clusters in SSE plot?
How to determine the number of clusters from the following plot?
- 1
2
votes
0
answers
37
views
Does it make sense to transform a feature containing hours (24h) into two features with xy-coordinates of each hour in the space? [duplicate]
I have a clustering problem that I might solve with an algorithm based on Euclidean distance (e.g. K-Means).
One potential feature is the "hour" at which each user began an interaction.
As ...
- 566
0
votes
0
answers
31
views
How to interpret the Scatter Plot result from PCA? [duplicate]
I have a project in school about clustering analysis. I have applied standardization and principal component analysis (PCA) to my dataset (I used K-means), which is about heart disease patients. I ...
- 1
1
vote
1
answer
140
views
In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)?
In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)? Can someone justify or give reasons to this statement?
Any guidance would be appreciated.
0
votes
0
answers
22
views
K-means on linearly projected features
I am looking for references on K-Means applied to linearly projected features instead of to the original features, in the sense that both K-Means and the projection matrix are learned at the same time....
- 213
1
vote
0
answers
48
views
Can K-means put most of the noise in the same cluster?
I am working on clustering text data (very short sentences) vectorized with tf-idf. The data are characterized by high sparseness and the presence of abundant noise (considered here as documents that ...
- 11
2
votes
0
answers
51
views
How can I ensure both levels of a binary variable are represented in every cluster?
Let's say I have some continuous variables and a binary treatment indicator. I want to cluster my observations based on the variables while ensuring that each cluster contains at least one member of ...
- 36.8k
2
votes
0
answers
370
views
How does this prove that the objective function in K-means clustering never increases?
I am reading the ISLR textbook (pg. 518-519, 12.4) and having trouble understanding why K-means clustering never increases. I can understand it conceptually but I don't understand the mathematical ...
- 21
2
votes
2
answers
1k
views
Time series clustering on large data
I am trying to do K-means clustering on my data which has time series length of 3700 and for (latitude,longitude) points of around 6000 in length. However, timeseries clustering using tslearn package ...
2
votes
0
answers
158
views
Normalization/standardization impact on T-SNE and K-means
I have a dataset of 20K samples on 27 features that I am trying to cluster with k-means. The dataset is in its majority rather sparse, i.e. 98% of samples have a single nonzero value in one of its ...
- 21
1
vote
1
answer
153
views
If I convert continuous data into ordinal discrete data, but number of the class is 100(1 class per 1 percentile), can I say this is 'continuous'?
I have some continuous data, and want to do kmeans clustering with this.
But weirdly when I did kmean clustering with this data, the outcome was very conflicted with my presumption. So I decided to do ...
- 23
0
votes
0
answers
28
views
Regarding kmeans clustering, is it ok if I set the number of clusters just to get appropriate number of data of target cluster that I'm interested in?
For example, let me suppose that I have 100 thousands of geospatial data in my dataset, and I want to extract certain group that which is the most crucial. So I decided to do clustering and pick 'one ...
- 23
1
vote
1
answer
79
views
Distance function that captures both circular and "appear as line" clusters [closed]
based on what I know in k-mean clustering, if i use single linkage distance it can capture clusters of thread shapes but it is not suitable for capturing circular clusters.
Also If we use complete ...
- 11
2
votes
2
answers
881
views
Jenks Natural breaks - Interpreting Goodness of Variance Fit
I am trying to find breaks in a multiple continuous type variables.
So, I tried the jenks natural breaks algorithm.
Based on the code from here, I managed to find ...
- 3,342
1
vote
1
answer
60
views
Does statistically simple algos qualify as AI algos?
We have a customer purchase transaction history data with variables like below
recency - how recently they bought?
frequency - How often they bought?
monetary - How much value did they bring to the ...
- 3,342
2
votes
1
answer
1k
views
Clustering leading to visually overlapping clusters on scatterplot
I am dealing with a dataset with 13 features.
After going through some standard scaling and missing data imputation, I use kmeans from sklearn to create clusters.
Now the point is that, although the ...
1
vote
0
answers
44
views
Differentiate between two set of points
Consider two sets of points (in the pictures below), whose "center of gravity" is same. What measure can differentiate between the two sets?
e.g.
Image 1 ...
- 161
0
votes
1
answer
109
views
Model based clustering equivalent to K means?
Is it OK to say something like this: "A model-based clustering with a hard threshold is equivalent to a k means clustering"? One of my instructors stated this in his slides, I kind of doubt ...
- 95
1
vote
1
answer
170
views
Features differ between classes
Good evening everyone.
Regarding the topic related to Sparse Clustering (for example K-Means).
For example, in "Witten DM, Tibshirani R. A framework for feature selection in clustering" the ...
1
vote
0
answers
99
views
Statistical method for finding homogeneous groups of curves
I need to divide a set of 100 or more response curves into groups.
These curves are formed by backscattering intensity along a range of frequencies. Basically, each curve represents the intensity in ...
- 11
1
vote
1
answer
286
views
Can we do clustering over several columns in a huge dataframe?
I have a dataset stands for customers retail sales data, it includes customer ID, Age, ...
- 339
1
vote
1
answer
604
views
Comparing clustering methods based on internal Cluster Validity Indices
I have used the R package dtwclust to generate clusters for more than a thousand time-series objects.Since I did not have any prior information on the number or validity of clusters, I used a suite of ...
- 41
0
votes
0
answers
62
views
Comparing clustering performance of two datasets?
For example:
Let's say I have dataset A:
Measured body temperature of a person during the day.
I have measurements from 3 people in the span of a year.
If I cluster it, I expect the clusters to ...
- 101
0
votes
1
answer
120
views
In R perform k means clustering with k=3 and euclidean distance a 100 different times [closed]
I would like to perform k mean clustering with k=3 and the Euclidean distance a 100 different time.
But it only gives me 2 iterations, how do i do a loop so it give me 100.
Thanks
- 3