Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [k-means]

k-means is a method to partition data into clusters by finding a specified number of means, k, s.t. when data are assigned to clusters w/ the nearest mean, the w/i cluster sum of squares is minimized

-2
votes
2answers
27 views

Normalization of Network data (clustering algorithms)

I have read in several academic articles that I can use clustering algorithms such as K-means to create clusters of network data. I have a dataset of IDS logs and I would like to create clusters ...
0
votes
1answer
20 views

K-means clustering scaling

I have a data set of 70 stores with a sales column (ranging from 50M to 70M) and 39 other features, like age group, income categories etc. I need to find the clusters based off of these metrics. A ...
0
votes
1answer
20 views

Clustering data multiple times to get statistics

Suppose I have some data (objects described by a number of variables like diameter, etc...) and that I want to run K-means clustering on it. Suppose now that I run the algorithm multiple times (1) on ...
1
vote
0answers
17 views

Apply K-means to the columns of the covariance matrix

In Section 5.3 of the paper distilling the knowledge in a neural network, it says we apply a clustering algorithm to the covariance matrix of the predictions of our generalist model, so that a set ...
0
votes
1answer
11 views

What does it mean for K mean problem to be NP hard and why?

Given a decision problem (a problem with yes or no answer), the problem is said to be NP-hard if there is an NP-complete problem Y, such that Y is reducible to X in polynomial time. Recall that NP-...
0
votes
1answer
19 views

How can we compute the difference between two silhouette scores for the same dataset?

Given a dataset X on which I applied k-means and I computed the Silhouette Index score. I consider this score as the truth. I applied again k-means on X and I computed the Silhouette Index score. My ...
3
votes
1answer
22 views

k-means clustered data: how to label newly incoming data

I have a data set with labels that were produced by a k-means clustering algorithm. Now there is some data (with the same data structure) from another source and I wonder what is the most sensible way ...
0
votes
0answers
40 views

Training a neural net as a classifier when there are no labels

I have an ML problem where I have a large data set and in this data set there are N categories. We have no labels. I want to be able to take this data set and use it to train a neural network to ...
0
votes
1answer
50 views

Clustering Data Using Gower and Kmeans

I am trying to do clustering on my data which consists of both categorical and continuous variables. I have some questions which I would like to ask: I am going to use the Gower Distance measure to ...
0
votes
2answers
33 views

Clustering Categorical Data

I want to cluster a data set where all variables are categorical. Which would be more effective for doing so, k - means or k - medoids? The data set is linked below. https://archive.ics.uci.edu/...
0
votes
2answers
26 views

What to do with small cluster size after k-means

So I use kmeans to 10k data with k = 8 as I took it from elbow analysis that will suggest me 5-8 cluster After the analysis, I got 1 cluster that only consist 1 member in it which I was not sure how ...
0
votes
0answers
21 views

Split an 1D array into N clusters but retain order

I am trying to split an array into N=6 parts which share some similarity but it is important that they retain the order they are in. An example is: ...
0
votes
0answers
11 views

There exists a k-means like method that allows good predictions when we have two sets of variables (one dependent and one independent)?

Suppose you have a set $X$ of dependent variables and a set $Y$ of dependent ones observed on $N$ individuals. So, I have a vague idea that a causal relationship should be validated(or measured, or ...
1
vote
0answers
12 views

k-means and (re?) standardisation of a sub-set

I have data which is customer purchases of items in each of three months: I have summed the data over the three months for each customer; calculated the proportion of purchases that each item ...
2
votes
0answers
22 views

Scalable kmean++ numerical example [closed]

I need a numerical example for computing the scalable kmeans++, since I'm not specialist in statistics and I didn't understand the messy greek letters in the algorithm. Any text reference link will be ...
-1
votes
2answers
34 views

K-Means clustering and correlation

I ran K-Means on my dataset, it's a small dataset of 200 countries x 6 export sectors. My results formed three clusters. Now I want to check whether these three clusters are correlated with another ...
2
votes
1answer
30 views

K - means, expected shape of the curve [closed]

I want to understand what happens as we increase the number of clusters using k- means, what is the expected shape of the curve showing the average distance between points and their assigned clusters? ...
-1
votes
1answer
30 views

Does this clustering quality metric make sense?

I am trying to stop at best quality metric in my clustering task. (I make spectral clustering using k-means). In short, I calculate intra-cluster pair-wise distances, take their square and sum them ...
-1
votes
1answer
7 views

How to Quantize Vectors using Kmeans?

I have a bunch of entities, with each instance having 40 features, so a 40-dimensional object. I cluster them using K-means. Now, I need to quantize them. I want to ask two questions: How to ...
0
votes
1answer
48 views

Clustering phrases using K-Means

I have a data set with some phrases. ...
0
votes
0answers
22 views

ANOVA (Type?) or Between Samples Test

Let's say I want to measure blood oxygen levels using three devices. Devices 1 and 2 are new devices and Device 3 is a gold standard that provides accurate readings. I'm interested not in diagnostic ...
0
votes
0answers
29 views

What are the advantages of Louvain method versus K-means for clustering sparse data?

I would like to better understand the strengths of the Louvain method versus K-means for high-dimensional sparse data (e.g. zero-inflated negative binomial gene expression counts or natural language ...
0
votes
1answer
36 views

How to inform the space and time complexity of K-means, SOM and Hierachical clustering

In the paper I am writing, one of the reviewers asked for an "a simple computational complexity analysis or time computational demands of their method" My question is : Can I simply report the ...
0
votes
1answer
17 views

How can i know that my dataset is being well distributed with K-means?

I'm trying to make an anomaly detection system using Spark Mlib an its K-means implementation but i'm struggling to decide when should i stop searching for K. I'm following Chapter 5 of the Advanced ...
0
votes
0answers
8 views

How to asses discoverability of a K-means based classification?

We are assessing a patent for "discoverability" (when a competitor copies the idea in the same domain with a similar approach we can say that is so just reviewing the customer facing feature). The ...
0
votes
2answers
51 views

Identifying k_means clusters number programmaticaly

I am stuck on finding a way to determine the elbow point (the optimal number of clusters to be used) programmaticaly. I need to run k-means on a set of 2D points obtained from an image and group ...
0
votes
1answer
32 views

Can I perform KMeans on a bimodal data?

I am preparing a dataset for KMean clusters. But a series of data appears to be bimodal: My question is: Can I perform KMeans on a bimodal data? If not, what kind ...
0
votes
1answer
26 views

Normalizing variables before clustering

I am looking to apply k-means clustering on two features of remote sensing data. The first layer is the Normalized Difference Vegetation Index (NDVI), which is expressed on a scale between 0-1. The ...
0
votes
0answers
113 views

Why can't K-means be used on LDA output

I started working on a topic definition task and my initial approach was as follows: Use LDA (Latent Dirichlet Allocation) to obtain the initial topic distribution for each of my documents. Then use ...
0
votes
0answers
14 views

Which is the best strategy for clustering a glossary of terms by exploiting their definitions

I have a glossary (dictionary) of terms together with short definitions (1 to 20 words). I want to cluster these terms by their application domains by using their definitions; for example, clustering ...
0
votes
0answers
12 views

Effect of dimensionality for time taken to cluster data with k-means

In a dataset if I have $N$ features and for k-means clustering it might take $T$ seconds. If the dimensionality increased to $2N$, how would the time taken to run k-means clustering increase?
0
votes
1answer
25 views

Which clustering algorithm is most suitable for grouping by set overlap?

I'm trying to cluster sets by their similarity in terms of included elements. The group of possible elements is of size ~1 million. It is my understanding that in order to run k-means or a similar ...
-1
votes
2answers
44 views

how to classify input image using clustering algorithm such as k-mean?

I want to classify cifar10 images using a clustering algorithm (k-mean). Each image in the cifar10 dataset has a label, so, the results must be a set of labels which are corresponding to the test ...
0
votes
1answer
62 views

Questions about a k-means variant : recompute centroids after each point is reasigned

I have a variant of k-means, where the points are reassigned incrementally and I have a few questions about it. Each time we reassign a point (we move the point from cluster $C_1 $to $C_2$), we ...
1
vote
1answer
48 views

A proof of within-cluster sum of squares?

Anyone can provide a proof of the following equation as in @cardinal 's answer? $x_i$ and $x_j$ are vectors from the same clusters。 $\sum_{i,j} ||x_i - x_j||^2 = \sum_{i \neq j} ||(x_i - \bar{x}) - (...
0
votes
1answer
97 views

K means clustering time fluctuates with increased value of K

I have written k means clustering code in c#. I am clustering random 99 text articles of Sports Area which I downloaded from Github for different values of K i.e.3,4,5,6,7. I want to analyze the time ...
0
votes
0answers
10 views

Is it appropriate to predict a trained KMeans model on holdout data that would not be included in the training set?

I have a KMeans model that is trained on features that are percentage-transformed descriptions of events. Each observation contains between 1 and 180 events. To help with meaningful comparisons, I ...
0
votes
0answers
17 views

Best method to obtain representative sample for clusters in high-dimensional space

I am clustering a large amount of high-dimensional data using KMeans (and the Euclidean distance metric), and then calculating the silhouette score and the Euclidean distance to the calculated cluster ...
0
votes
1answer
36 views

Best Clusterizing Techninque for 7 points Likert scale

What is the best clustering method seven points Likert Scale. When what I am looking to answer is if there are groups of people behavior on it. For example. I have around 30 questions with this scale....
1
vote
1answer
49 views

What does it mean to apply k-means algorithm on transformed distance matrix?

I am reading a very good (recent) publication in clustering: Kiselev et al., 2017, SC3 - consensus clustering of single-cell RNA-Seq data (if you don't have access, see author PDF). The algorithm ...
0
votes
2answers
25 views

Unsupervised Classification of Linear Trends

I have a data set which results in a series of non-parallel linear trends on a scatter plot. I'm trying to find a way to classify each data point into its closest corresponding linear trend. There ...
1
vote
1answer
325 views

How to I determine the maximum number of iterations in K-means clustering?

In the documentation of kmeans, the default value of iter.max is 10: kmeans(data, modes, iter.max = 10, weighted = FALSE, fast = TRUE) I don't understand why. And I also wonder how to determine the ...
0
votes
1answer
145 views

K-Means Variable Selection

I have a simple data set, 1200 Rows and 20 variables, 1 is a categorical variable with 8 unique values. 1 variable is a unique reference number. I'm looking into using Kmeans clustering to find the ...
0
votes
1answer
52 views

Clustering for medium data [closed]

Which clustering method is good in R for a data with ~32,000 subjectsa and 10 variables, hierarchical or k-means?
2
votes
2answers
285 views

Best BIC value for K-means clusters

I am using code from Using BIC to estimate the number of k in KMEANS (answer by Prabhath Nanisetty) to find BIC values for K-means using different number of components. However, using iris dataset, I ...
0
votes
1answer
154 views

Silhouette score behaving counter intuitively

My Silhouette score decreases as number of clusters increase. I'm using scikit's kmeans algorithm on the modified white wine dataset from UCI. Here's the final dataset I'm using - https://drive.google....
2
votes
1answer
124 views

How can I order kmeans clusters?

I have a kmeans cluster object and I would like to order the clusters. Not the observations within the clusters, rather the clusters in order of each other. Is there a way of doing this? I found ...
0
votes
1answer
22 views

Kmeans results, is the cluster vector ordered by 'closeness"?

I ran kmeans in r with k = 20 centers and 7 scaled variables to cluster with on a data frame with n = 100K. Using dplyr group_by I was able to view summary data for each of the 20 clusters: the mean ...
-1
votes
1answer
103 views

How k-means computes cluster centroids differently for each distance metric?

K-means computes cluster centroids differently for each distance metric. I don't know why the way of computing the centroid is dependent of the distance measure. I don't know how we compute the ...
0
votes
1answer
18 views

How to interpret contrasting information from the Variation of Information, Dunn and Rand Index for comparing clusterings

There are related questions but the answers don't seem to explain how to practically judge these measurements for non stats users. I have a dataset which I clustered with K=4 using hierarchical ...