Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [k-means]

k-means is a method to partition data into clusters by finding a specified number of means, k, s.t. when data are assigned to clusters w/ the nearest mean, the w/i cluster sum of squares is minimized

0
votes
1answer
34 views

K-Means Algorithm

Struggling with getting a mathematical proof for K-means. Can someone help ? Let D(d) be the average distance to centroids in K-means with k centroids . If we denote D*(k) as the average distance of ...
0
votes
0answers
7 views

What does a high silhouette score for assigning everything to 1 cluster mean?

I'm writing my bachelor's thesis and I'm running into an oddity. When running k-means and hierarchical, the clustering is fairly evenly distributed - there isn't a clear preponderance of data points ...
0
votes
0answers
29 views

Detecting seasonal anomalies using k-means [closed]

I have a huge network log file which contains messages from all different devices in the LAN network. A number of devices send periodic/cyclic messages - some messages are sent every hour, some are ...
0
votes
1answer
15 views

How normalize a list of time series features with pyspark?

My goal is to normalize a list of time series to perform a kmeans clustering My dataset is a dataframe with hashtags as entries and column containing time serie features like: ...
1
vote
1answer
35 views

K-means calculate MSE in Weka

I am doing some clustering analysis with Weka and decided to apply the k-means algorithm (the clusterer SimpleKMeans). On my first analysis I ran the algorithm with 2 clusters. Then, after finding ...
-1
votes
2answers
29 views

Initialization of Kmeans++ clusters

I would like to confirm the initialization steps of my K-means++ implementation (steps which chose initial centers of clusters). I am wondering if my initialization scheme has been implemented ...
0
votes
2answers
29 views

R - high dimension data using k means clustering [closed]

The dataset is 1000(observations) x 700(variables), After using pca to do dimension reduction, PC150 explained 85% Variance, so I use this (1000 x 150) data to do k means clustering. This code was ...
0
votes
0answers
16 views

Guidance on analysing my PCA plot / NLP methodology,

I have been analysing documents which also contains chapters. I use TF-IDF to generate the word embeddings and then take the cosine similarity of a document chapter at year ...
0
votes
0answers
29 views

High dimensional clustering (K-means and DBSCAN)

My research is all about comparing the K-means and DBSCAN(Density-Based Spatial Clustering with Application of Noise) and I used python with the aid of jupyter notebook. I have 28 variables and 3048 ...
-1
votes
2answers
28 views

Unsupervised Clustering

My research is about comparing K-means and DBSCAN, and Im using unsupervised learning method in clustering. Is it true that the number of cluster in K-means is also the same number as the unique ...
0
votes
0answers
8 views

How to find anomalies/outliers in Panel Data (Unsupervised)?

I have panel data based on 900000 different entities with 384 time steps and the data is not normally distributed. I am looking for outliers/anomalies, this is unsupervised as I have no examples of ...
0
votes
1answer
22 views

How to select a single updated centroid if multiple centroids are equidistant for a single group when running k-means/k-medoids?

I am trying to write my own k-means and k-medoids clustering algorithms. I understand the general idea: given k centroids, one continually updates the centroids ...
0
votes
0answers
11 views

K-Means vs Rocchio

I'm new to machine learning and data science. I am currently learning about K-means and Rocchio. From what I've gathered so far it seems that Rocchio can be a form of K-Means. Is that right or am I ...
0
votes
0answers
21 views

Time series novel

I've exhaustively attempted to find a proper way to analyse a dataset. Despite finding several piece of information of what could be done, I kindly ask for suggestions of could be done, mainly in R. ...
-1
votes
2answers
45 views

Alternative method to k-means that can be “guided” researcher's intuition? [closed]

I am trying to do a simple k-means clustering to my dataset. The result I get it the one that can be seen below: However, the result I would like to have, as it corresponds to geographical areas, ...
1
vote
2answers
76 views

In k-means clustering, why sum of squared errors (SSE) always decrease per iteration?

In k-means clustering, why sum of squared errors (SSE) always decrease per iteration? How can prove it by mathematical derivation of formulas? k : number of clusters m : number of examples $c_h$ : ...
0
votes
2answers
22 views

How to include percentage variables in PCA + K-means when some values are undefined because the denominator is 0?

I'm trying to do customer segmentation by using PCA to reduce dimensionality and then feeding the resulting principal components into a K-means algo to get at the final segments. Some of my variables ...
1
vote
0answers
13 views

Graph clustering for balanced sum of absolute deviations within each cluster (same sum of intracluster distances)

I'm given a set of points and a distance matrix. With these I'm trying to develop an algorithm similar to k-means that tries to minimize the sum of distances from each cluster datapoint to it's center ...
0
votes
1answer
23 views

How can K-Means clustering work without spatial information?

Just got stuck at working with K-means clustering. I have looked up this python/skimage commands: ...
3
votes
0answers
29 views

Is my variable considered okay to use in k-means clustering with Euclidean distance?

I was wondering if I can use regular kmeans() in R with my variable "number of drug prescriptions" which equals a number between 1-25. From what I've read k-means ...
0
votes
0answers
19 views

Obeservation Weighted KMeans

I have an unbalanced textual dataset. Now when I perform K-Means clustering on that with the prior information that the dataset needs to be divided into two cluster(Elbow analysis and Shilloute Score),...
0
votes
1answer
27 views

Can Silhouette score compare algorithms based on different metrics?

If I intend to compare the clustering performance between K-Means and K-Modes clustering using this measure. How do I do so? y data set is binary in nature and I want to see if K-Modes using Manhattan ...
0
votes
0answers
117 views

How to determine the best batch-size value for Mini Batch K-means algorithm?

I am working on a project where I apply k-means on severals datasets. These datasets may include up to several billion points. I would like to use mini batch k-means to save time. However, the mini ...
2
votes
1answer
23 views

Relation between pairwise distance sum and sum of distance to mean (gap statistic)

I'm trying to understand the gap statistic used for optimal choice of $k$ in k-means clustering. I'm trying to understand part of the explanation which includes this equality: $D_k=\sum_{ij}\Vert x_i-...
-2
votes
2answers
41 views

Normalization of Network data (clustering algorithms)

I have read in several academic articles that I can use clustering algorithms such as K-means to create clusters of network data. I have a dataset of IDS logs and I would like to create clusters ...
0
votes
1answer
78 views

K-means clustering scaling

I have a data set of 70 stores with a sales column (ranging from 50M to 70M) and 39 other features, like age group, income categories etc. I need to find the clusters based off of these metrics. A ...
0
votes
2answers
27 views

Clustering data multiple times to get statistics

Suppose I have some data (objects described by a number of variables like diameter, etc...) and that I want to run K-means clustering on it. Suppose now that I run the algorithm multiple times (1) on ...
1
vote
0answers
21 views

Apply K-means to the columns of the covariance matrix

In Section 5.3 of the paper distilling the knowledge in a neural network, it says we apply a clustering algorithm to the covariance matrix of the predictions of our generalist model, so that a set ...
0
votes
1answer
19 views

What does it mean for K mean problem to be NP hard and why?

Given a decision problem (a problem with yes or no answer), the problem is said to be NP-hard if there is an NP-complete problem Y, such that Y is reducible to X in polynomial time. Recall that NP-...
0
votes
1answer
26 views

How can we compute the difference between two silhouette scores for the same dataset?

Given a dataset X on which I applied k-means and I computed the Silhouette Index score. I consider this score as the truth. I applied again k-means on X and I computed the Silhouette Index score. My ...
3
votes
1answer
32 views

k-means clustered data: how to label newly incoming data

I have a data set with labels that were produced by a k-means clustering algorithm. Now there is some data (with the same data structure) from another source and I wonder what is the most sensible way ...
0
votes
0answers
41 views

Training a neural net as a classifier when there are no labels

I have an ML problem where I have a large data set and in this data set there are N categories. We have no labels. I want to be able to take this data set and use it to train a neural network to ...
0
votes
1answer
117 views

Clustering Data Using Gower and Kmeans

I am trying to do clustering on my data which consists of both categorical and continuous variables. I have some questions which I would like to ask: I am going to use the Gower Distance measure to ...
0
votes
2answers
59 views

Clustering Categorical Data

I want to cluster a data set where all variables are categorical. Which would be more effective for doing so, k - means or k - medoids? The data set is linked below. https://archive.ics.uci.edu/...
0
votes
2answers
34 views

What to do with small cluster size after k-means

So I use kmeans to 10k data with k = 8 as I took it from elbow analysis that will suggest me 5-8 cluster After the analysis, I got 1 cluster that only consist 1 member in it which I was not sure how ...
0
votes
1answer
33 views

Split an 1D array into N clusters but retain order

I am trying to split an array into N=6 parts which share some similarity but it is important that they retain the order they are in. An example is: ...
0
votes
0answers
12 views

There exists a k-means like method that allows good predictions when we have two sets of variables (one dependent and one independent)?

Suppose you have a set $X$ of dependent variables and a set $Y$ of dependent ones observed on $N$ individuals. So, I have a vague idea that a causal relationship should be validated(or measured, or ...
1
vote
0answers
15 views

k-means and (re?) standardisation of a sub-set

I have data which is customer purchases of items in each of three months: I have summed the data over the three months for each customer; calculated the proportion of purchases that each item ...
2
votes
0answers
23 views

Scalable kmean++ numerical example [closed]

I need a numerical example for computing the scalable kmeans++, since I'm not specialist in statistics and I didn't understand the messy greek letters in the algorithm. Any text reference link will be ...
-1
votes
2answers
75 views

K-Means clustering and correlation

I ran K-Means on my dataset, it's a small dataset of 200 countries x 6 export sectors. My results formed three clusters. Now I want to check whether these three clusters are correlated with another ...
2
votes
1answer
56 views

K - means, expected shape of the curve [closed]

I want to understand what happens as we increase the number of clusters using k- means, what is the expected shape of the curve showing the average distance between points and their assigned clusters? ...
-1
votes
1answer
37 views

Does this clustering quality metric make sense?

I am trying to stop at best quality metric in my clustering task. (I make spectral clustering using k-means). In short, I calculate intra-cluster pair-wise distances, take their square and sum them ...
-1
votes
1answer
7 views

How to Quantize Vectors using Kmeans?

I have a bunch of entities, with each instance having 40 features, so a 40-dimensional object. I cluster them using K-means. Now, I need to quantize them. I want to ask two questions: How to ...
0
votes
1answer
61 views

Clustering phrases using K-Means

I have a data set with some phrases. ...
0
votes
0answers
25 views

ANOVA (Type?) or Between Samples Test

Let's say I want to measure blood oxygen levels using three devices. Devices 1 and 2 are new devices and Device 3 is a gold standard that provides accurate readings. I'm interested not in diagnostic ...
1
vote
0answers
127 views

What are the advantages of Louvain method versus K-means for clustering sparse data?

I would like to better understand the strengths of the Louvain method versus K-means for high-dimensional sparse data (e.g. zero-inflated negative binomial gene expression counts or natural language ...
0
votes
1answer
62 views

How to inform the space and time complexity of K-means, SOM and Hierachical clustering

In the paper I am writing, one of the reviewers asked for an "a simple computational complexity analysis or time computational demands of their method" My question is : Can I simply report the ...
0
votes
1answer
18 views

How can i know that my dataset is being well distributed with K-means?

I'm trying to make an anomaly detection system using Spark Mlib an its K-means implementation but i'm struggling to decide when should i stop searching for K. I'm following Chapter 5 of the Advanced ...
0
votes
0answers
9 views

How to asses discoverability of a K-means based classification?

We are assessing a patent for "discoverability" (when a competitor copies the idea in the same domain with a similar approach we can say that is so just reviewing the customer facing feature). The ...
0
votes
2answers
57 views

Identifying k_means clusters number programmaticaly

I am stuck on finding a way to determine the elbow point (the optimal number of clusters to be used) programmaticaly. I need to run k-means on a set of 2D points obtained from an image and group ...