All Questions
53 questions
0
votes
0
answers
19
views
Clustering Mixed Data Types: Algorithm Selection, Distance Measurement, and Feature Weighting
I have a database of 74,000 records with 29 features. Fourteen of these features are categorical and are either 0 or 1, while the other 15 features are continuous and have been normalized and scaled ...
1
vote
1
answer
60
views
Elbow method not giving a proper curve in python code
I am trying to determine how many clusters to use for my k-means clustering using different methods.
first i used the following code to calculate different metrics per cluster number and different ...
- 11
4
votes
2
answers
411
views
Question about Silhouette index calculation using scikit
I am currently working with continuous data measured from different sensors (thermometers and voltmeters). I have a matrix whose columns represent the sensors and the rows are normalized measurements (...
- 205
1
vote
1
answer
112
views
How to tell whether segments from K Means clustering result are "successful" and will impact business metrics?
Background
I'm a data analyst. The Business unit I'm assigned for needs to segment users based on power vs non-power users so they can target each segment with proper treatments.
Goal
Segment users (...
- 135
1
vote
1
answer
1k
views
Elbow method Vs Gap statistics, which one? challenging for data scientist
I am working on hourly-weather data. It contains four features: rain, wind speed, humidity, and temperature. Obviously, all of them are continuous values. The number of records is around 17000. Other ...
- 73
1
vote
2
answers
93
views
Can I use K-Means to group customers based on a single variable?
I have a test dataset of 11m records. The dataset contains a global customer id and spend figure.
I need to group customers into the following categories:
0 Low
1 Low/Med
2 Med
3 Med/High
4 High
I ...
- 11
0
votes
0
answers
19
views
How do I choose k for k means clustering [duplicate]
Given a set of points, I'm trying to find the right cluster. However, I am lost on what the process is. Here is the graph of all possible points.
I am unsure what I should look at
whuang2017
0
votes
1
answer
108
views
Time Series clustering: clustering a dictionary of time series
I'm working on classifying times series to find clear pattern of use. My data is collected from clients of a telecom company, and we want to detect pattern of the amount of data consumed by clients ...
0
votes
0
answers
78
views
cluster 2d matrix (clustring)
i have a 2d Matrix and It contains specifications for laptops, where each group contains three components like thant :
...
1
vote
1
answer
346
views
Clustering Data with Time and ~10 million records
I have a dataset with features like product categories, their dimensions, price, units sold on a given day.
I want to create clusters out of this dataset (~12-15 million records) and I am using data ...
1
vote
1
answer
377
views
standardization/normalization for 1D clustering?
I have two input variables revenue and age. Am trying to find different bins within that variables.
For ex: I have ...
- 3,342
0
votes
1
answer
2k
views
How to evaluate unsupervised Anomaly Detection using k-means
I'm trying out different anomaly detection models and would love to hear opinion on my idea from somebody experienced. My goal is to perform anomaly detection with different models and to give each ...
- 3
2
votes
2
answers
179
views
How to find the number of clusters when more than one datasets are aggregated as one?
Suppose 3 datasets has 3 ,7, 4 clusters in their respective dataset.
When I aggregated them as one dataset what's the safest number of cluster to choose as perimeter for kmeans or any supervised ...
- 23
0
votes
1
answer
177
views
Transformation of features in KMeans when maximums and minimums are different?
I have some questions about KMeans that I would like to discuss.
I have several features, the minimum and maximum values between columns vary, so I applied the "MinMaxScaler" ...
- 101
0
votes
1
answer
184
views
The resulting image file is even larger than original when using K-means to do image compression
I am trying to compress jpeg file
[Original Picture]
[Compressed Picture with K-means using K=10]
However, the original one is 85K while the compressed one is 101K?
Here is the code I use:
...
- 11
1
vote
0
answers
172
views
I don't understand why each time Kmeans finds the same centroid given different initialization?
The k-means algorithm does the following:
Given a set of points, we first choose k random points to be the initial centroids.
We then create k clusters. The ith cluster contains the points nearest to ...
- 11
2
votes
0
answers
281
views
Perform clustering from a similarity matrix
I have a list of songs for each of which I have extracted a feature vector. I calculated a similarity score between each vector and stored this in a similarity matrix. I would like to cluster the ...
1
vote
1
answer
97
views
Find all possible clusterizations
I need help to find all possible clusterizations via the k-means method in Python. Let's assume for simplicity that I have the following table:
height | weight | country of origin (X/Y/Z) | flag (1/0)
...
- 131
1
vote
2
answers
595
views
K-Means output the similar to each other cluster
I am trying to run K-Means on my data set of house price prediction problem.
After running it, the output of the model seems wrong because the graphs look the same as each other.
This is my code:
<...
- 113
0
votes
0
answers
33
views
k means in python with BIC [duplicate]
I am new to ML. I am trying to implement k-means which uses a BIC function that takes cluster and data points as arguments and returns BIC value. I need a function to find best k value that is ...
- 101
1
vote
0
answers
36
views
Finding centroids without K-means
The Data:
Currently I have a simple distribution that looks like the histogram below. Every point is an integer between 0 and 16 and I have 350MB of these samples.
The Problem:
I want to identify 8 ...
- 131
0
votes
1
answer
199
views
How to evaluate the perforamance of clustering model using python
I have implemented the k means clustering model using python , i would like to know whether my model is perfect or not , so that i want to know which performance metrics is used for clustering model ...
- 23
0
votes
0
answers
232
views
Which is the best clustering algorithm for clustering multidimensional data with low density difference?
I am working on a project currently and I wish to cluster multi-dimensional data. I tried K-Means clustering and DBSCAN clustering, both being completely different algorithms.
The K-Means model ...
5
votes
2
answers
1k
views
For K-means clusters, how can I ensure each cluster has a minimum of n numbers
I usually use k-means++ for initialization, which is considered to be the most effective.
But sometimes, this results in some clusters having too few constituents.
While this may be mathematically ...
- 71
1
vote
0
answers
162
views
python find the optimal # of cluster for K-Means algorithm
I have a data that contains 24 features and all features have some missing values. I want to use the impute-KNN algorithm from sklearn to fill the missing values. However, before I do that, I think I ...
- 21
0
votes
1
answer
85
views
Visualize Analysis of clustering after pca
I am using kmeans for clustering and if I read the topics around here and somewhere else it is always recommended to do a graphical check-up for the number of ...
- 236
1
vote
1
answer
359
views
Should I scale when clustering text data using K-means?
I want to cluster a folder of texts. I created a data file where for each text, I write whether a certain word appears in it or not. I want to cluster according to this.
So my matrix is globally only ...
- 363
1
vote
1
answer
151
views
Modifying k-means for points on torus
My data coordinates are degrees so each axis has values [-180, 180]. Therefore it's easy to spot that in fact the scatter plot on the right end continues on the left side and the same thing for up and ...
- 61
0
votes
2
answers
656
views
K-means which normalization fits
Hi am working on a business dataset, where I want to group the participant in k-means based on some features.
The problem is I have to create this features upfront, so that I combine different ...
- 236
0
votes
1
answer
526
views
How to interpret the different cluster sizes in Silhouette plot?
I created silhouette plots for my clustering models by following: this link
I want to know what does the different cluster sizes mean and how they were generated?? I understand that thicker size ...
- 141
0
votes
1
answer
154
views
k-means clustering issue voice data
I'm getting an issue in my k-means I don't know if it my data-set or what anything else.
Why i got thia flowing point in the right side of the image?
...
- 115
0
votes
0
answers
252
views
Using k-means clustering to train radial basis neural network for highly imbalanced dataset
I am trying to find prototype neurons for my radial basis neural network. My dataset has 30 attributes (of which 28 of them are the result of a single PCA) and 300.000 observations. It is a binary ...
- 3
0
votes
2
answers
382
views
Unsupervised Classification of Linear Trends
I have a data set which results in a series of non-parallel linear trends on a scatter plot.
I'm trying to find a way to classify each data point into its closest corresponding linear trend. There ...
- 103
1
vote
2
answers
4k
views
Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?
I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like:
Since this was ...
- 159
0
votes
0
answers
367
views
Weighted K-means for my super market vs K-means
I have a Super Market. I want to find if product A is out of stock which product should i replace with. I am not sure what should i do, someone suggested me K-means for that. If sppose my data looks ...
Shyam Kishor
0
votes
1
answer
428
views
Formatting input data to Scikit learn for Kmean and PCA
I am very confused about the data that feeds to Kmean and PCA algorithm using Scikit Learn command in Python. I searched a lot in the internet but no where I found the clear answer.
I have $X$, a $m \...
- 281
1
vote
0
answers
2k
views
Correct calculation of BIC (Bayesian Information Criterion) to determine K for K-Means
I am trying to calculate BIC in python. In python, there is no inbuilt library for computing BIC. I referenced the following link to compute variance and BIC further:- Using BIC to estimate the number ...
- 111
0
votes
1
answer
10k
views
Calculating clusters Entropy, Python
I ran K-means++ algorithm (Python scikit-learn) to find clusters in my data (containing 5 numeric parameters).
I need to calculate the Entropy. As far as I understood, in order to calculate the ...
- 141
1
vote
0
answers
35
views
Identify Phrases Not in Training Set (Unsupervised)
I'm trying to train an unsupervised machine learning algorithm to learn a vocabulary and if it is given a word, it can predict how close that word is to what it already knows. Only issue is my ...
- 111
0
votes
0
answers
252
views
Theano K Means with Shared Variables and Scan
I have a pet project to reproduce some common clustering in theano in order to improve my understanding for future projects. I was wondering if anyone has ever used nested theano scans on shared ...
- 1
3
votes
0
answers
342
views
How to deal with variability in clustering. Multiple/Meta clustering?
I'm not sure what information is relevant here, so here is some background:
I'm using Python 3 / sklearn, but I could probably use R if needed.
I have a small sparse data-set (~1500 samples, ~1600 ...
- 191
2
votes
0
answers
420
views
A problem with implementing PCA-guided k-means
I am new to machine learning. I am reading the papers K-means Clustering via Principal Component Analysis and PCA-guided search for K-means. But there are too many mathematical proofs in these papers. ...
- 21
14
votes
4
answers
23k
views
How to avoid k-means assigning different labels on different run?
I have unlabeled dataset. I am running k-means flat cluster with 2 number of clusters. Every time I run the below program the labels are different. How can I make labels not to change. Is it even ...
- 345
0
votes
1
answer
2k
views
How to cluster an 1-D array by K-means or any other algorithm using scikit-learn? [closed]
I have an one dimensional toy array X.
I want to cluster the data into some numbers of clusters.But when I try to fit my data in scikit-learn K-Means function it shows
ValueError: n_samples=1 ...
- 105
1
vote
1
answer
3k
views
K-Means Clustering Not Working As Expcected
I have a script that I'm testing with in Python3 with Scikit to cluster terms based on either words or character n-grams. Basically, it's fed a list of training data with corresponding labels. For ...
- 205
0
votes
0
answers
528
views
I want to classify data by distance from centroids in python
I'm making an image classifier that will tell if an image is a car or not, in Python.
here are my steps:
Get SIFT descriptors from about 200 images with cars on them.
On all those SIFT descriptors ...
- 21
0
votes
0
answers
1k
views
What could cause a K-means clustering algorithm to converge into a single cluster?
I am currently writing a K-means clustering algorithm in Python, and I seem to have coded myself into a corner... I begin my algorithm with data sets assigned randomly to the appropriate number of K ...
- 1
35
votes
4
answers
50k
views
Clustering a correlation matrix
I have a correlation matrix which states how every item is correlated to the other item. Hence for a N items, I already have a N*N correlation matrix. Using this correlation matrix how do I cluster ...
- 483
17
votes
4
answers
36k
views
Using BIC to estimate the number of k in KMEANS
I am currently trying to compute the BIC for my toy data set (ofc iris (: ). I want to reproduce the results as shown here (Fig. 5). That paper is also my source for the BIC formulas.
I have 2 ...
- 540
12
votes
1
answer
32k
views
Clustering inertia formula in scikit learn
I would like to code a kmeans clustering in python using pandas and scikit learn. In order to select the good k, I would like to code the Gap Statistic from Tibshirani and al 2001 (pdf).
I would like ...
- 812