Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Methods for Data Reduction

Subject: Methods for Data Reduction in Emission Rate Calculations Dear Community, I have conducted numerous measurements in a barn to calculate annual emission rates (such as ammonia, methane, etc.). ...
Didi_Agri's user avatar
-1 votes
0 answers
38 views

How to prove that K-Means splits all space into convex polygons?

I want to prove that K-Means algorithm splits the whole object space into convex (probably not bounded) polygons. I've tried to use the fact about convergence of K-Means algorithm and get ...
John Doe's user avatar
0 votes
0 answers
8 views

Discovering a natural grouping structure in data

I'm looking to apply some data using the sparse-group lasso. This method requires that the variables sit within groups, so I need to pass group labels to the model. Is there an efficient method for ...
user19904's user avatar
  • 125
1 vote
0 answers
22 views

1 dimensional autoencoder as a clustering tool?

I am looking for references (as I prefer to sit over the giant's shoulders...) to something it "seems" to work well... When we do clustering to analyse some data, to understand its structure ...
Antonello's user avatar
  • 371
0 votes
0 answers
14 views

How to evaluate performance and optimize hyperparameters for clustering algorithms on a dataset with continuous labels?

I'm working on a clustering problem where my dataset's labels are continuous numerical values, not discrete categories. I'm using t-SNE and UMAP to reduce the dimensionality of my dataset's features ...
Jason Shi's user avatar
0 votes
1 answer
7 views

How to Define Equipment Churn in Laboratory Service Data Without Explicit Churn Labels?

I'm working with a comprehensive dataset spanning 20 years of service records for laboratory equipment owned by various customers. This dataset captures intricate details, such as the equipment ID, ...
tlengman's user avatar
2 votes
0 answers
38 views

Problem with mathematical formulas in gap statistic

I'm studying the article "Estimating the number of clusters in a data set via the gap statistic" by R. Tibshirani, G. Walther and T.Hastie: https://academic.oup.com/jrsssb/article/63/2/411/...
user2702's user avatar
2 votes
1 answer
31 views

What is the standard threshold value that is best for accuracy when employing Euclidean distance as a metric for gauging textual similarity?

I'm using Euclidean distance as a metric to compare two sentences for similarity while clustering them using my custom incremental KMeans algorithm. The current threshold value I'm using is 0.7 which ...
sanjay M's user avatar
1 vote
1 answer
30 views

Inference from profile comparisons

What would be a good test to compare patterns of differences among treatments? For example, the Y maze has an outcome of % of entries into each of three arms. Thus, three numbers are generated for ...
Bryan's user avatar
  • 1,212
1 vote
1 answer
31 views

Equivalent of gap statistic for k-medoids on non-Euclidean spaces

I have a connected graph with weighted edges. I want to partition the graph into communities with a clustering algorithm. I chose K-medoids and I run it on a distance matrix, where ...
miguelmorin's user avatar
0 votes
0 answers
11 views

What is the standard performance metric for categorical data clustering?

I performed a categorical clustering with some selected UCI datasets. I one-hot encoded the features, then directly used Binomial Mixture Model and KModes using this one-hot encoded data. On the ...
NOT-A-CS-GUY's user avatar
0 votes
0 answers
16 views

Stock clustering based on fundamental reporting

I want to made a stock clusterization, based on their fundamental features from companies quarter reports. I collected quarter reports from 2018 to 2022. Some companies have reports for all quarters ...
TImur Nazarov's user avatar
0 votes
1 answer
36 views

Problems Determining Optimal Cluster Number for Time-Series Data

I'm facing problems with determining the right number of clusters for my 2D time-series data. I have a numerical simulation that outputs a time-series of 2D grids that represent a mass density ...
Double Descent's user avatar
0 votes
0 answers
15 views

How can one measure spatial clustering of continuous (non-count) data?

I am currently teaching an ecology class how to use the variance to mean ratio (VMR) as a method for looking at the spatial clustering of individual organisms across a landscape. This method takes the ...
Michael L's user avatar
  • 101
2 votes
0 answers
53 views

Finding parameters which reveal clustering in t-SNE

These data are from SAMHSA, Mental Health Client-Level Data. I am trying to find the right parameters to obtain clustering as in this paper. Code here. For now, I'm dropping columns which aren't ...
Jackson Walters's user avatar
0 votes
0 answers
10 views

What is normalized winning frequency in kernel self organizing map(SOM)?

In the k-means based kernel SOM, proposed by MacDonald and Fyfe (2000), the update of the mean is based on a soft learning algorithm mi(t + 1) = mi(t) + Λ[φ(x) − mi(t)] where Λ is the normalized ...
Anshuman Jayaprakash's user avatar
0 votes
0 answers
31 views

Why does this K-Means cluster example show 'overlap' between clusters?

I was reading the hypertools docs and came across this pictorial that shows 10 clusters (some seem to share very similar coloring) generated from some (mushroom) ...
Vincent Karuri's user avatar
0 votes
0 answers
9 views

Distance to find similar samples in a multivariate dataset

Apart from Euclidean and Mahalanobis distance metrics. Given a sample with multivariable values, is there a way to find the samples that are similar to the given sample? Does KNN clustering find the ...
sveer's user avatar
  • 103
0 votes
0 answers
19 views

Question on the proof step in the theorem 1 of the Gap statistic paper

From the Gap statistic paper, during the proof for the theorem 1, we can see the below equality (p. 422), $\begin{aligned} \operatorname{var}(X) & =\frac{1}{2} \int_{-\infty}^{\infty} \int_{-\...
kurtkim's user avatar
  • 303
0 votes
0 answers
33 views

Is it valid to use unsupervised clustering to assign patients into larger groups before comparing them to controls?

I have very high-dimensional data (lipidomics) with two categories: patient/control and genetic mutation. These mutations have similar phenotypes, but they are considered different diseases. Their ...
maglorismyspiritanimal's user avatar
1 vote
0 answers
58 views

K-means clustering - weird PCA visualization

I performed PCA on 4 variables and are shown in this visualization: At first look it doesn't look convincing and the some clusters seem weird. The data was cleaned and standardized beforehand. Only ...
Simon's user avatar
  • 11
0 votes
1 answer
33 views

Clustering of large text datasets with unknown number of clusters

I have a list of hotel names which may or may not be correct, and with different spellings (such as '&' instead of 'and'). I want to use clustering in order to group the hotels with different ...
user480840's user avatar
0 votes
0 answers
37 views

Clustering an ordinal variable with unequal interval size

I am interested in clustering a mixed type dataset and therefore I have found the package "clustMixType" in R Link to package description. I want to cluster numerical, categorical and ...
ExchangedVisual111's user avatar
1 vote
0 answers
18 views

Finding the most-similar color palette [closed]

I have a dataset of several thousand color palettes: for each row in a dataset, I have the ten most common RGB values in an image, and the fraction of that color in the overall image: ...
Saul Aryeh Kohn's user avatar
4 votes
1 answer
181 views

how to compute clustering error properly

true = c(1,1,1,2) pred1 = c(1,1,2,2) pred2 = c(1,1,2,3) Suppose my dataset has two clusters, after using two clustering algorithms, one gives pred1 and the other ...
Simple's user avatar
  • 207
0 votes
0 answers
24 views

Variable Clustering (varclus) R-squared from SAS in R

I am using the varclus function from the Hmisc package in R. Are there ways to produce summary tables from varclus like those in ...
mautumn1's user avatar
0 votes
1 answer
19 views

Categorical Intra-Cluster Quality - Suggestions

I've derived some clusters from categorical data using a few of different methods - k-medians, k-modes, k-medoids, etc etc. I've used validation-style methods for the determination of optimal k. The ...
EB3112's user avatar
  • 244
0 votes
0 answers
12 views

ML Clustering with an added condition

Problem: I want to create distance-based clusters of customers where each cluster, in sum, yields the same revenue potential. Explanation: I'm looking at thousands of customers spread throughout a ...
Tommy Lee's user avatar
0 votes
0 answers
21 views

Method for pairwise ordering two datasets

Given two rather small but unordered multidimensional vectors/datasets (e.g sets of a handful of 3D coordinates), what is a simple method for pairwise alignment/ordering? I've though about using ...
joaocandre's user avatar
0 votes
0 answers
9 views

Comparing two datasets of communication flows

I know the question of how to compare two datasets has been asked multiple times and I have read a lot about the different tests available but I'm a little lost on what exactly the tests show and what ...
Milan D's user avatar
  • 101
0 votes
0 answers
31 views

Why do my optimal number of clusters changes everytime I re-run the same code? Is this normal?

I have the following code: ...
Emilia's user avatar
  • 1
0 votes
0 answers
15 views

How to get a site-adjusted (clustered) p-value

I'm working on a problem involving clinical trial data and am trying to account for the different clinics patients were treated at. In papers I've read, authors present p-values for differences in ...
smirza's user avatar
  • 21
0 votes
0 answers
16 views

Multiple comparisons (i.e., cluster 1 vs cluster 2+3), how to adjust p-value

I am analyzing the differences in mean of a continuos variable between 3 clusters. I used anova (after testing whether it has a parametric distribution) and obtained p-value lower than 0.05. As post-...
user406121's user avatar
0 votes
0 answers
31 views

Elbow method not giving a proper curve in python code

I am trying to determine how many clusters to use for my k-means clustering using different methods. first i used the following code to calculate different metrics per cluster number and different ...
rebwar's user avatar
  • 1
0 votes
0 answers
7 views

Diagnostic checks before and after adjusting for standard errors in R

Currently, I'm fitting a regression model for my dataset. As there are clusters, I refitted the model using the coeftest() function from the ...
iGada's user avatar
  • 217
0 votes
0 answers
19 views

Clustering and segmented regression interrupted time series analysis

I would like to perform an interrupted time series analysis to look at the impact of the pandemic on cancer incidence using individual level data. I plan to use a negative binomial segmented ...
user405452's user avatar
1 vote
0 answers
23 views

Outliers in Delta Time column, using data from wireshark

I am currently analyzing data downloaded from Wireshark, focusing on real-time network traffic. I need to perform clusterization on this dataset. However, during the visualization process, I noticed ...
Jaminka's user avatar
  • 11
2 votes
1 answer
36 views

Using Anova with count variables

I am testing a hypothesis where my independent variables (Xs) are count data ranging from 0 to 10. My dependent variables (Ys) are also count data, ranging from 0 to 10. I am expecting to see an ...
NutellaMonster's user avatar
1 vote
0 answers
51 views

Select closest individuals with primary and secondary criteria

I have a set of samples divided in 4 different groups that will be compared by RNA-seq later on. My objective is to select the closest individuals possible, all groups confounded, based on qualitative ...
Basti's user avatar
  • 61
1 vote
0 answers
24 views

Evaluate two datasets for corresponding to given pairwise matching

Suppose I have two matched $N \times D$ datasets $X$ and $Y$. The samples are in matched order (with $n$th sample in $X$ being the same object as the $n$th sample in $Y$). The features are (or at ...
Betterthan Kwora's user avatar
0 votes
0 answers
18 views

Hierarchical cluster analysis with mixed data

I have a dataset consisting of 134 observations and four variables. The dataset consists of answers to a questionnaire. I want to perform a hierarchical cluster analysis on the variables in the ...
Lasse H's user avatar
0 votes
1 answer
74 views

Hierarchical cluster analysis with likert scale ordinal variables

I have a small dataset of 134 observations. The dataset consists of answers to a questionnaire. All the variables are measured in Likert-scale, ranging from 1 to 5 (strongly agree). I want to perform ...
Lasse H's user avatar
1 vote
0 answers
12 views

Adjusted Rand Index Per Single Cluster

i have a nuanced ARI question for which i can't find any reference. say i have two clusterings, say ClusteringA and ClusteringB, and i am comparing them using ARI. now, i'm not interested in the ARI ...
GiAmit's user avatar
  • 11
2 votes
1 answer
57 views

Should I use strata(), cluster() or frailty() to account for center specific characteristic in a time-varying analysis

The research question revolves around whether a time-varying intervention at a center enhances the transplantation hazard per person. I have patient-level data and I need to account for the center ...
Claudio Bravo's user avatar
0 votes
1 answer
30 views

Continuous monitoring of KMeans model post production

In the process of deploying a KMeans model for a customer segmentation use case into production. KMeans doesn’t produce the same results every time and after production cluster sizes and arrangements ...
ibarbo's user avatar
  • 65
0 votes
0 answers
49 views

Examples of UMAP performing poorly

Can anyone suggest some datasets or examples that would be useful for showing situations where UMAP performs poorly, say in the sense of producing spurious clusters, or missing clusters? For t-SNE, ...
Tom Solberg's user avatar
1 vote
1 answer
35 views

Availability of Linear Grouping Algorithms to Linearly Cluster Datasets

I have been trying to cluster a scatter plot that has a triangular graph, ideally the proper clustering plot should have a linear form, as shown below: I tried using Spectral Clustering: and ...
NOT-A-CS-GUY's user avatar
1 vote
1 answer
33 views

What is the most appropriate index for categorical data clustering?

I am trying to replicate a study published by Bai & Liang, 2022 which focuses on clustering purely categorical data which are mostly found in the UCI repository. In my experiment for K-Modes, I ...
Gerard's user avatar
  • 181
0 votes
0 answers
46 views

Instance segmentation using a discriminative loss?

I have been reading this paper and I was wondering if their discriminitive loss definition is correct for instance segmentation ? From what I understand they map the image pixels into a higher ...
KFkf's user avatar
  • 111
0 votes
0 answers
24 views

Drawing a dendrogram based on a confusion matrix

I have a simple 2D confusion matrix that I normalised for every true class. Now, what would be the best method to visualise this as a dendrogram? I have seen hierarchical agglomerative clustering on ...
Damiaan Reijnaers's user avatar

1
2 3 4 5
81