Questions tagged [k-means]

k-means is a method to partition data into clusters by finding a specified number of means, k, s.t. when data are assigned to clusters w/ the nearest mean, the w/i cluster sum of squares is minimized

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

How to avoid having very different examples inside a cluster with Kmeans?

Let's say I've created some clusters with Kmeans using 5 features, the Silhouette Score for these cluster are very high, higher than 0.8, and The within-cluster sum of squares is around 130 in this ...
user avatar
0 votes
0 answers
20 views

Which clustering methodologies are likely to be best for this data?

I'm using the classic "use-case" example of clustering pixels in a photograph. I've tried K-means, agglomerative clustering, and DBSCAN. When I plot the RGB coordinates in 3-D space, all 3 ...
user avatar
1 vote
0 answers
9 views

Correlated variables and similar loadings in first principal component

I'm doing a K-Means model for first time, thus very low experience. I read that it is not bad to discard variables through some PCA analysis. After standardizing the data, the loadings (weights) for ...
user avatar
0 votes
0 answers
9 views

Spatial clustering with maximum group weight

I am looking for a clustering method that would allow spatial clustering of a set of points (with weights associated to each point) with maximum cohesivity, where each group of points must have at ...
user avatar
2 votes
1 answer
36 views

Confusion on why the value of SSE is lower when a cluster looks distorted on the plot

I have a dataset of shape (29088, 11). When I apply the Kmeans where K=2 I get the following plot: Cluster C0 has 8554 points (in blue) and cluster C1 has 20534 ...
user avatar
  • 151
0 votes
0 answers
18 views

Clustering algorithms that support FA rather than PCA

In our social sci research we've used Factor Analysis rather than PCA. It would be helpful for us to use a clustering algorithm to group respondents into the most logical factor groups. Kmeans seems ...
user avatar
  • 1
0 votes
0 answers
45 views

How should I determine the optimal number of clusters in kmeans in R with large data set?

I am right now dealing with a large dataset and I have used kmeans and fviz_nbclust in R with wss method to try to determine the optimal clusters in k-means clustering. But as the figure shows, the ...
user avatar
  • 1
0 votes
0 answers
28 views

K-Means clustering should cluster data evenly distributed or unevenly distributed? [duplicate]

I am clustering customers using their stay time on our web sites. When I only use one variable, time, for K-Means clustering with 10 clusters, customers look unevenly distributed to each clusters. ...
user avatar
  • 2,585
0 votes
0 answers
26 views

How to choose a fair gamma value when performing k-prototypes clustering?

In the k-prototypes clustering algorithm, the distance function consists of two dissimilarity components - one for the numerical elements of the observations, and one for their categorical elements. ...
user avatar
0 votes
0 answers
9 views

CH-index continues to fall instead of peaking

I'm doing a clustering exercise with k-means algorithm. The natural number of clusters of the dataset is 5. I'm testing number of clusters between 2 and 15, but CH-index keep decreasing with the ...
user avatar
0 votes
0 answers
24 views

Shapley values for the three clusters by cluster number KMeans algorithm

I am trying to replicate this https://cast42.github.io/blog/datascience/python/clustering/altair/shap/2020/04/23/explain-clusters-to-business.html#Kmeans-clustering But using R and not Python as in ...
user avatar
  • 1
0 votes
0 answers
11 views

How to generate states for a Markov-Model through K-Means-Clustering on time-series?

I am reading a paper by Zufferey et al.: https://ieeexplore.ieee.org/document/8442470 On page 2 it says: "In this paper, the definition of the states is based on a K-Means clustering which allows ...
user avatar
0 votes
0 answers
19 views

Customer segmentation by category

I have a dataset where I must perform customer segmentation. The dataset columns are organized as follows: Franchise Store | Attribute 1 | Attribute 2 | .... | Attribute N The unique elements in "...
user avatar
0 votes
0 answers
8 views

analytically determine how many clusters you need to get an explained variance of over x%

I am currently trying to cluster my data with as few clusters as possible. I have tried using K-means clustering and spectral clustering. Both work relatively well, around 85% explained variance from ...
user avatar
0 votes
0 answers
41 views

K means clustering on long format data

I have a dataset of customers and some of their characteristics, including the total worth of goods purchased. Essentially, I have transaction line items. My data looks something like this: ...
user avatar
  • 45
0 votes
0 answers
17 views

Any good method to identify the principal components on a low dimension timeseries dataset?

I have a low dimension (only 8 dimension) Time Series dataset of simple z-score standardized numerical data. Number of raw data point is about 2000 x 8. I tried running PCA on the dataset by splitting ...
user avatar
  • 63
1 vote
1 answer
69 views

Use K-means clustering on SVD/PCA of data

In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and ...
user avatar
1 vote
0 answers
20 views

Does it make sense to use variables' product as a new variable in a clustering procedure?

I'm trying to separate different groups based on values from width and length using k-means and hierarchical clustering. My question relates to the possibility of using the area — measured as width * ...
user avatar
1 vote
0 answers
14 views

K means clustering analysis to define geological facies using 2 attributes (ERT & seismic)

dear all. Currently I am doing a project where the goal is to define geological facies of an area by using ML. The method that we are doing is k-means (we have no labels beforehand) and we are using ...
user avatar
  • 11
2 votes
0 answers
54 views

interpretation of elbow plots [duplicate]

Hi I have this elbow plot that was created to select the K for clustering but I can't find a sound explanation of how to interpret this, all I ever see is a picture of an elbow with in plots with ...
user avatar
2 votes
0 answers
73 views

PCA explained variance and model inertia

I'm trying to perform a PCA to reduce the dimensionality of my data and subsequently perform a K-Means algorithm. I initially chose 4 Principal Components because they explain 70% of my variance. This,...
user avatar
2 votes
1 answer
57 views

Scaling a power law distribution for k-means clustering

For my project I want to group some products by using a few variables. For grouping, I am using k-means clustering. One of my variables is a metric called CR (conversion rate) which takes values ...
user avatar
0 votes
0 answers
28 views

How to kmeans to clustor datetime by period weakly, montly and yearly

I have a list of bills with id, merchant name and paid date, I want to cluster this data based on the period this bills are paid, meaning the outcome result of the clustering should group the bills ...
user avatar
0 votes
1 answer
19 views

Need help choosing appropriate algorithm for building a Lookalike audience

Newly practicing data scientist here! I am currently stumped on a project and reaching out for some guidance: I am working with the marketing team in our customer database. There is a small subset of ...
user avatar
  • 1
0 votes
0 answers
52 views

Select number of clusters k-means

I have a practical question. I am trying to select the number of clusters in k-means clustering and I have tried a Silhouette analysis, an elbow plot looking at the residuals, and a hierarchical ...
user avatar
0 votes
1 answer
89 views

Clustering data points with multiple rows

I have 100 people with their mobile browsing records, where each record tracks the person's browsing url and duration etc., and thus each person will have multiple rows of records. Now I want to ...
user avatar
0 votes
0 answers
50 views

How to save a Higher accurate K-means Model on a unlabelled data based on Any Performance Evaluation Metrics?

I am experimenting on Iris dataset. I am not using the label. I want to save my model based on any Performance Metrics. According to Performance Metrics which model have higher score I am choosing ...
user avatar
  • 51
4 votes
2 answers
174 views

Why did Clustering Algorithms Become so Popular Despite their Results often Being "Uninterpretable"?

I understand the clustering algorithms are usually considered as "unsupervised algorithms", which means they can function in the absence of a response variable, making them applicable in ...
user avatar
  • 5,688
3 votes
0 answers
34 views

Cluster Algorithm for multidimensional data

My goal is to cluster data (20000 samples with a range from 0.0 to 1.0, and 14 dimensions/features). Since I don't know the number of clusters, I tried using MeanShift and DBSCAN. My problem with ...
user avatar
0 votes
0 answers
24 views

How to cluster data based on temperature values and time

I'm having an issue analyzing data of oil temperature over time (vehicle engine oil). My goal is to detect the parts where the temperature cools down over time in this "exponential decay" ...
user avatar
  • 1
0 votes
0 answers
15 views

When does K-Means give High Quality Clusters?

It's well known k-means algorithm is effective when the clusters are spherically or elliptically distributed. This data assumption is not true in many real world data examples. Are there real-world ...
user avatar
0 votes
0 answers
54 views

Does it make sense to use the decoder of LSTM autoencoder as an input to K-means clustering?

I have a large time-series data - with 200 (time) variables. I want to use a clustering algorithm to cluster my data based on the values of (200) time variables. Before using a clustering algorithm ...
user avatar
  • 123
0 votes
0 answers
79 views

Interpret cluster analysis results with simulated data in R

Generate a dataset with 300 observations and three variables: f, x1, and x2. f should be a factor with three levels, where level 1 corresponds to observations 1-100, level 2 to 101-200, and level 3 to ...
user avatar
  • 1
1 vote
0 answers
30 views

Clustering large yearly, (presence/absence) dataframe

I have a data frame of 500,000x23 dimensions. The data is binary, representing presence or absence. The data follows identified trees through time (23 years) and looks at whether the tree is present ...
user avatar
1 vote
1 answer
57 views

Identity for K-Means Clustering

The property (12.18) from here states that $$\frac{1}{|C_k|} \sum_{i, i' \in C_k} \sum_{j = 1}^{p} \left(x_{ij} - x_{i'j}\right)^2 = 2 \sum_{i \in C_k} \sum_{j = 1}^{p} \left(x_{ij} - \frac{1}{|C_k|} \...
user avatar
0 votes
0 answers
30 views

Understanding Kmeans prediction

I am using the Kaggle heart disease dataset and using Kmeans to understand the clustering of the data. I used several of the continuous variable columns and the target (age, maxHR, cholesterol, ...
user avatar
  • 1
0 votes
0 answers
32 views

Clustering and T-test (number of neurons in two groups of mice)

I will present my problem to you. I have a database of brain slides of 40 mice. There are two groups of mice, mice 1 and mice 2, and each of the groups is made up of 20 mice. On each slide of the ...
user avatar
0 votes
0 answers
21 views

Evaluating K-Means with Percentage Variance Explained on non-normal data

Hi have Poisson distributed data which I am running an unsupervised K-Means algorithm on. I was doing some research on this site, and found this page which discusses K-Means on non-normal data, ...
user avatar
  • 1
1 vote
2 answers
64 views

K-Means results interpretation when we have no idea of the number of clusters

I have a dataset with 11 variables and 80 000 observations. I know 2 techniques to find evidence of clusters in a dataset: hierarchical clustering and k-means. I can't use the hierarchical clustering ...
user avatar
1 vote
1 answer
311 views

Is k-means a generative model and how could it be used to generate new data then?

In today's lecture we learnt that k-means would be generative model. I am really puzzled on this because in my intuition it would be more a discriminative model since there is no probability to ...
user avatar
0 votes
0 answers
16 views

How to apply a grouping algorithm to time series data?

I have a time series data set with 4 variables and 2 known groups. The 2 groups are "Equities" and "Commodities". I have used R to create the 2 groups where the correlation and ...
user avatar
0 votes
0 answers
18 views

Clustering and data scaling

I have a dataset with 5 questions, which are scaled 1-10 and income variable, which is nominal. Should I standardise all variables with min/max scaler, or convert income to 1-10 scale? What is the ...
user avatar
  • 1
0 votes
0 answers
44 views

1D clustering, 1<=k<=3, known ~ratio between mean of each k, what to do?

I have data that should be split into 1-3 clusters. For each cluster, the respective means are x, 3x, and ...
user avatar
0 votes
0 answers
55 views

How to optimize K-means to eliminate outliers and unrelated clusters?

I clustered document embeddings with K-Means. Embeddings have 2048 dimensions. Now, i am trying to optimize clustering. There are two problems. 1- Some clusters may have outlier samples. 2- Sometimes,...
user avatar
0 votes
0 answers
49 views

How to interpret result of kMeans scores if I have encoded the data with OneHotEncoder?

I am working on the AdventureWorks database and I have extracted some demographic data from the person scheme as follow. My aim ...
user avatar
4 votes
1 answer
429 views

K Means as a special case of GMM (using EM Algorithm)

I am looking for a tutorial/gentle introduction (preferably with mathematics/proofs) on K-means as a special case of Gaussian Mixture Model using the EM Algorithm. I have found this: https://www....
user avatar
  • 7,545
0 votes
0 answers
34 views

What effect would clusters get when adding more variables to clustering task?

I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
user avatar
  • 2,585
1 vote
0 answers
165 views

Difference between Hamming Loss, Hamming Score, and Hamming Distance in multiclass multilabel classification

I am trying to understand the mathematical difference between, Hamming distance, Hamming Loss and Hamming score. I am trying to perform two actions Multiclass multi label classification using SVM K ...
user avatar
0 votes
1 answer
107 views

K means formula in argmin

$$ \underset{m_1,m_2,\dots,m_k}{\operatorname{arg\,min}} \sum_{i=1}^n \underset{j=1,2,\dots,k}{\min} \| \mathbf{x}_i - m_j \|^2 $$ I found this equation but I forgot the source, but what I remember is ...
user avatar
1 vote
0 answers
26 views

K-Means clustering technique for monthly data

I have an Unsupervised problem where user's Credit Card payment data is given for each month for various users for one year. One of the feature in the data having "User Id". For most of the ...
user avatar

1
2 3 4 5
20