Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

Filter by
Sorted by
Tagged with
1
vote
0answers
17 views

How to prove that trangle inequality is satisfied in Hausdorff distance [migrated]

I'm working on a problem my teacher asked me to check if I was interested, which is 'how to prove that Hausdorff Distance is strictly a distance function'. More specifically, how to prove that $$D_H(A,...
0
votes
0answers
9 views

In clustering, how does one estimate the distribution over the number of clusters?

In Murhpy's Machine Learning book pg 10, he says, when introducing clustering (Given unlabelled data) Let K denote the number of clusters. Our first goal is to estimate the distribution over the ...
0
votes
0answers
13 views

Data association on data from multiple cameras

Suppose we have several cameras that cover a certain area. In each camera we track a person. Each person have a path in global coordinates, timestamps and a feature-vector. The goal is to group these ...
1
vote
0answers
11 views

Number of min clusters to set in hcpc function within FactoMineR [closed]

I am doing MCA prior to clustering (both using FactoMineR package). I was wondering if there is any possible way to determine the min number of clusters to set in the code?
0
votes
0answers
17 views

How can i do Clustering using KL Divergence? [closed]

I just wanna know how KL-divergence can be used for clustering data.
0
votes
1answer
24 views

Objective Statistic for Comparing Clusters

I wondered if anybody knew of any objective measures for comparing clusters on ordinal data? For example, suppose I was to run a standard clustering routine such as partitioning around medoids/k-...
0
votes
2answers
26 views

Numerical and Categorical variables in datasets [closed]

When trying to understand and analyze huge datasets that contain categorical and numeric variables, what methods can I use to cluster, or more generally, view in a meaningful way to find correlation. ...
0
votes
1answer
13 views

Guidance with piecewise linear data set

I have data that looks like: As you can see my linear modeling doesn't really work since the y values increase and then stay constant. I want to separate my data for each group into 2 and then ...
0
votes
2answers
26 views

Can not define the right number of cluster on K-Means

so i am a newbe on K-means, i use some methods to identify the number of cluster that i can use, but i found out there is some different output on each method. 2 clusters on silhouette, 8 clusters on ...
1
vote
1answer
23 views

How to cluster univariate time series

I have different univariate time series and the goal is to detect outliers automatically. Therefore I used different algorithms for different time series. But the first step would be to detect ...
1
vote
0answers
19 views

Non-negative matrix factorization clusters

NMF can be used for clustering i.e., $V=WH$ where $W$ represents cluster centers and $H$ represents the membership of samples. But can NMF alone cluster the samples? Can we get better clusters in NMF ...
1
vote
2answers
32 views

K-Means results interpretation when we have no idea of the number of clusters

I have a dataset with 11 variables and 80 000 observations. I know 2 techniques to find evidence of clusters in a dataset: hierarchical clustering and k-means. I can't use the hierarchical clustering ...
0
votes
0answers
19 views

Good clustering method for compact clusters

Suppose I have a LARGE dataset with unknown labels (around 40000 points) where the pairwise Euclidean distance does not differ too much ( mostly 11- 13, some 6-9). Which clustering algorithm should I ...
2
votes
1answer
30 views

Is Nonnegative matrix factorization a clustering method or a dimensionality reduction method?

In the matrix factorization we have the problem of decomposing a nonnegative matrix $X$ into two lower-rank matrices $W$ and $H$. I would like to know whether this method is considered as a dimension ...
0
votes
0answers
12 views

How to apply a grouping algorithm to time series data?

I have a time series data set with 4 variables and 2 known groups. The 2 groups are "Equities" and "Commodities". I have used R to create the 2 groups where the correlation and ...
0
votes
0answers
16 views

Clustering and data scaling

I have a dataset with 5 questions, which are scaled 1-10 and income variable, which is nominal. Should I standardise all variables with min/max scaler, or convert income to 1-10 scale? What is the ...
0
votes
0answers
8 views

Shop Classification based on two metrics

I'm trying to do a classification analysis in order to classify group of merchants based on their total transaction amount (in USD) and also their total transaction (how many transactions). Notes: ...
0
votes
0answers
33 views

1D clustering, 1<=k<=3, known ~ratio between mean of each k, what to do?

I have data that should be split into 1-3 clusters. For each cluster, the respective means are x, 3x, and ...
0
votes
1answer
62 views

Silhouette was not returning a valid number on scikit-learn on iris data. Is this wrong?

I was testing some clustering validity indexes with Iris Dataset and I got something odd with scikit learn. The silhouette index is giving a better index for 2 clusters instead of 3 clusters (the real ...
1
vote
1answer
39 views

How to deal with visually overlapping clusters

I have done hierarchal clustering (average) and silhouette method shows 6 as the best k. However visualising I see that most of the clusters are overlapping. Any way to deal with the overlapping ...
0
votes
0answers
37 views

How to optimize K-means to eliminate outliers and unrelated clusters?

I clustered document embeddings with K-Means. Embeddings have 2048 dimensions. Now, i am trying to optimize clustering. There are two problems. 1- Some clusters may have outlier samples. 2- Sometimes,...
0
votes
0answers
23 views

Silhouette coefficients with random data

I have point pattern data for the distribution of animals in fields. I'm hoping that the community can confirm (or soundly dismiss!) my newbie understanding of the silhouette method. Would it be the ...
0
votes
0answers
17 views

questions about the use of clustering in social science

I'm a student in social science. Recently i'm learning clustering, but I really can't understand how can we use clustering in our social science research. For example, I use the 'gender consciousness' ...
0
votes
0answers
26 views

Negative silhouette width in clustering

I have some clusters having observations below 0 Si (showing that they are not in a right cluster I assume). I am doing hierarchal clustering on a binary data. One of the solutions I found was ...
0
votes
1answer
19 views

Robust Sparse Clustering: how do we choose L1 and alpha parameters?

Before asking, I would like to say that I was not sure where to ask this question - on CrossValidated or on StackOverflow. Finally, it seems to me more logical to ask it here. So, I've read an article ...
0
votes
0answers
18 views

Multiple correspondence analysis by groups (patterns of violence data)

I am interested in doing some MCA to identify some patterns of violence data (observations by events and variables such as perpetrator, type of violence, against whom, where). I would like to identify ...
0
votes
0answers
25 views

Which unsupervised clustering technique to identify potential high-value zones from volume profile?

Ideally, would like to implement an unsupervised approach (ultimately in C#) that is a starting point to define prospective high-value supply and demand zones from a multi-day composite volume profile....
0
votes
0answers
14 views

Using Classification using User Profile for Product Recommendation

I want to build a ML model which will recommend products to users (Currently I have 3 products to recommend A,B and C) So I have some user profile data, like address, gender, age, and few other ...
0
votes
0answers
20 views

What effect would clusters get when adding more variables to clustering task?

I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
0
votes
1answer
39 views

Clustering methods for regional temporal clusters

I have observations of individuals over time, where they either experience -1, +1 or mostly 0, like so. ...
0
votes
1answer
47 views

What is the optimal number of clusters?

I am doing a cluster analysis with agglomerative hierarchical clustering on my asymmetrical binary data. For finding the number of clusters, I tried all three of the most mentioned methods (Elbow, ...
0
votes
0answers
20 views

Distance measure for ordinal data (multiple measures)

I have collected user ratings for 4 different ordinal variables. Variable A has (likert) scale from 1-7 but can be collapsed to 1-3, Variables B, C, D have (likert) scale from 1-4. For each item, x (a ...
0
votes
0answers
15 views

Hyperparameter selection in Affinity Propagation without ground truth

My goal is to implement affinity propagation for clustering a given dataset (n=12 features), and I wish to find the optimal hyperparameter value (preference) allowing an educated guess of the number ...
0
votes
0answers
60 views

Can the Response Variable ever be used in Clustering?

I have the following question: Can the response variable ever be used in a clustering algorithm? I understand that in general, clustering is considered to be an "unsupervised learning algorithm&...
0
votes
0answers
15 views

Relating a scoring mechanism to entropy and homogeneity

Context I'm working on some clustering algorithms, and my boss wants me to grade them. My go-to method has been to measure homogeneity, completeness, and v-measure. However, since we are presenting ...
0
votes
0answers
29 views

Evaluation of non-symmetric clustering algorithms

I have the following dataset comprising of 4 different densities, presented visually in the photo below Using Kmeans these are the results of my classification, which obviously are not good. Using ...
1
vote
0answers
32 views

HDBSCAN: most data clustered as noise (-1)

I am trying to perform topic modeling on text data, ie. cluster the text messages by topic. I am approaching this by using a BERT model to get sentence embeddings, then use T-sne to reduce the ...
1
vote
1answer
38 views

Transform variables with zero-inflated values and positive skewness

I have over 30 features: several have zero-inflated and highly positive skewed distribution. Those distributions are expected because they are semi-continuous monetary related features. For example: ...
1
vote
1answer
62 views

Obtain a between-class similarity. And is the way to do it through PCA valid?

Context: I have a dataset containing instances labeled into different classes, and for all the classes, I have the same set of features. My research question is to identify classes that are more ...
0
votes
1answer
33 views

K means formula in argmin

$$ \underset{m_1,m_2,\dots,m_k}{\operatorname{arg\,min}} \sum_{i=1}^n \underset{j=1,2,\dots,k}{\min} \| \mathbf{x}_i - m_j \|^2 $$ I found this equation but I forgot the source, but what I remember is ...
0
votes
0answers
24 views

How to measure similarity between two sets?

I want to measure the similarity between sets of images. Every set contains unique image names. The biggest challenge for me is that the sets are not equally sized, and not all images of setA are in ...
4
votes
1answer
33 views

Can Decision Trees be used to Identify Clusters ("Cohorts") within the Data?

I have the following question: Can Decision Trees be used to identify Clusters ("Cohorts") within the Data? I present my question in the context of Survival Analysis Regression (using the R ...
1
vote
1answer
36 views

How to evaluate the accuracy of clustering text data

I have text data from customer inquiries, and want to figure out what are the main topics customers enquire about. I am approaching this by using a pre-trained BERT SentenceTransformer model ('...
2
votes
0answers
86 views

Combining variables and PCs to find clusters

I am working on a 100 variables dataset, and I am following a recipe for calculating clusters: Select only variables that have less correlation, or are below a 10 value in VIF Then for the unselected ...
0
votes
0answers
18 views

Doubts about generating a synthetic dataset according to a paper

I'm trying to replicate the experiment reported in section 3.3 of this paper (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2930825/) but I'm struggling to understand how the synthetic dataset is ...
2
votes
1answer
29 views

cosine similarity with weights

We're doing pairwise similarity computation for some real estate properties. Our data goes something like this: ...
0
votes
0answers
32 views

Computing BIC and other methods to find most likely K cluster

I have recently been diving into the world of population genetics and still remain with some questions when trying to conclude what the most likely value of K for my dataset would be. To give you some ...
1
vote
1answer
83 views

How to compute a 'pair confusion matrix'?

I don't really understand how the pair confusion matrix (used for example in comparing of clusterings) is calculated... ...
0
votes
0answers
18 views

A "multidimensional d" to include uncertainty in distance matrices

We're all familiar with Cohen's d, difference between means divided by pooled standard deviation. Is it valid to incorporate the concept to create a "deviation adjusted distance matrix"? ...
0
votes
0answers
19 views

Silhouette score: counter-intuitive results

so I was looking back at this tutorial (https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html) and it struck me that the example with ...

1
2 3 4 5
76