Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

How to best bucket data to optimize performance of models?

I'm training multiple models to predict churn for customers belonging to online stores. Each model contains customer-level features and data related to n-number of stores. However, we've paid barely ...
user avatar
0 votes
0 answers
12 views

Measuring Similarity of Multidimensional Time Series

Suppose I have a non-linear time series comprised of 100 timesteps, within each I have 4 features for each of 50 observations. The features are not independent of eachother and the relationships ...
user avatar
0 votes
0 answers
3 views

Given a set of mus and standard deviations is it possible to cluster Gaussian distributions?

Are there any methods for clustering multiple Gaussian distributions? So give x mus and x standard deviations can you create clusters of these Gaussians? My preliminary thought is just to check which ...
user avatar
  • 129
0 votes
0 answers
13 views

Standardization before clustering is always a good idea? [closed]

I'm performing k-means clustering based on 5 features with different scales and Mean > Median. So, I scaled the data before applying kmeans. The Average silhouette width is therefore maximized for ...
user avatar
  • 1
0 votes
0 answers
10 views

Cluster evaluation - evaluation separation, stability and agreement - best indices?

In the literature many indices exist that aim to quantify the level of separation of the clustering outcome: Silhouette, Davis-Bouldin, Calinski-Harabasz, etc. To evaluate the stability of a ...
user avatar
0 votes
0 answers
11 views

Cosine Similarity of the word embeddings after UMAP dimensionality reduction

I want to calculate similarities of the word embeddings. As a basis I took SpaCy german corpus: nlp = spacy.load("de_core_news_lg"). I have approximately ...
user avatar
0 votes
0 answers
10 views

How to detect edge blindness and within-cluster blindness

I am trying to solve a problem by clustering where I am using OPTICS as well as DBSCAN algorithm for clustering my latitude/longitude data with Phaversine distance metric. But in the course of the ...
user avatar
1 vote
0 answers
28 views

Each Principal Component Has Low Variance Explained

I have a survey dataset with 200 columns (encoded as numbers) and am trying to reduce the number of dimensions. After applying PCA, I can reduce the number of dimensions but each PC barely explains ...
user avatar
  • 51
0 votes
0 answers
17 views

K-prototype in R: Error when including missing values

I want to cluster data that includes categorical (dummies and variables with multiple categories) and numerical variables (normalised) and a substantial amount of missing values. One reason why I want ...
user avatar
  • 1
0 votes
1 answer
11 views

Time Series clustering: Changing warping window for Dynamic time warping

I'm working on the same type of data and i want to classify the times series to find clear pattern of use. My data is collected from clients of a telecom company, and we want to detect pattern of the ...
user avatar
0 votes
0 answers
6 views

Neutralizing the effect of variables before a clustering analysis

Working on a research project, I have 10 performance scores (scales, ordinates variables) for 100 participants. In addition, I have the age, gender (1/0) and education years of participants as ...
user avatar
  • 1
0 votes
0 answers
10 views

Noisy Data & Model-based Clustering [closed]

I have two noisy data sets (N1 = 100k posts, N2 = 30k posts) coming from a discussion board that I'd like to deduce themes from. I've processed and used embedding strategies that reduce the noise (ex. ...
user avatar
2 votes
1 answer
14 views

Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
user avatar
  • 21
0 votes
0 answers
14 views

Clustering trends across categories: which one to use?

I have a dataset with daily temperatures for every country in 4 seasons: Spring, Summer, Fall, and Winter. I wish to group(cluster) countries with similar temperature trends across the 4 seasons. I am ...
user avatar
0 votes
0 answers
15 views

How to interpret entropy of cluster? [closed]

Considering the label of model 1 as true label and model 2 as the predicted label. I calculated the entropy which got 0.331. Again consider the label of Model 2 as a true label and model 1 as the ...
user avatar
0 votes
0 answers
14 views

Optimality of KL-Divergence for Clustering Distributions

I have two related theory questions. In both cases, I imagine the following workflow: I see several datasets $D_{1},\ldots,D_{k}$, I transform these observations into estimated distributions $\mu_{1},...
user avatar
1 vote
1 answer
31 views

How to determine agreement between clustering methods?

Let's say you want to compare the outcome of KMeans and KMedoids. How to determine if cluster 1 from KMeans can be compared with cluster 1 with KMedoids. Or, in other words, let's say KMeans labels ...
user avatar
0 votes
0 answers
6 views

Clustering: using labels to improve cluster on second unlabeled dataset

Imagine I have two datasets $D_1$ and $D_2$ both datasets have the same formats: They consist of $n$ and $m$ points respectively and each and these points form $n_c$ and $m_c$ clusters (which I don't ...
user avatar
1 vote
1 answer
30 views

Clustering Data with Time and ~10 million records

I have a dataset with features like product categories, their dimensions, price, units sold on a given day. I want to create clusters out of this dataset (~12-15 million records) and I am using data ...
user avatar
1 vote
0 answers
40 views

Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?

Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density $$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
user avatar
  • 11
0 votes
0 answers
42 views

Hierarchical ranking/clustering algorithm?

Consider two situations: 1st situation You ordered a number of cubes by their weight (from light to heavy). Then you notice that most of the cubes are grey, but there're a lot of the cubes with ...
user avatar
0 votes
1 answer
17 views

How to justify anonymous clusters?

Is there a reasonable situation where the clusters are anonymous? What I mean is that one can ensure the subjects sampled are from the same cluster, but she does not know exactly which cluster they ...
user avatar
  • 51
0 votes
0 answers
11 views

K-Prototypes did not form clusters

I implemented a K-Prototypes algorithm (Huang) to cluster some mixed data in order to solve a customer segmentation question. There aren't a crazy amount of observations (n = ~6k) and with 8 fields (2 ...
user avatar
  • 103
1 vote
0 answers
24 views

hdbscan vs dbscan in how they cluster the data of varying shapes and density [closed]

Can you explain this passage please: "A key feature of HDBScan is that it clusters data of varying density, this is in comparison to DBScan, which tend to cluster data of varying shapes only.&...
user avatar
1 vote
0 answers
13 views

Clustering On Labelled Dataset [closed]

I've seen posts discussing clustering on labelled datasets. Below is an approach I'm considering. IN short, I would be clustering "base data" vs. "XAI-calculated" data to see how ...
user avatar
0 votes
0 answers
21 views

What is the objective function of complete linking clustering?

In complete linkage clustering, we start by the similarity matrix and we combine the elements with the highest similarity into a cluster. With this new cluster $(x_i,x_j)$, we calculate the new ...
user avatar
0 votes
0 answers
41 views

clustering approach to identify temporal "states" within time-series in R

I am working on a problem where I have a multiple time series, each with a size of a 100 steps, each being described by a 8 variables variables. I want to identify "states" within each time ...
user avatar
  • 121
0 votes
0 answers
8 views

making a column of multivariate time series data stationary, not the already stationary ones

I have a multivariate time series dataset. Two dimensions are not stationary. Is it technically correct to make these two dimensions stationary by differencing and leave the other dimensions intact, ...
user avatar
1 vote
1 answer
56 views

Applying k-means over PCA

I have a dataset containing 20 columns and 200 rows. This is an unlabeled dataset and I applied PCA to this dataset for dimensionality reduction. After successfully using PCA, I received a dataset ...
user avatar
  • 19
1 vote
0 answers
11 views

Visualizing and clustering user activity within tool ecosystem

I have a large amount of telemetry for a population of users using a tool suite. In many cases, users are working on a number of different tasks over the course of a day, and hopping back and forth ...
user avatar
  • 111
0 votes
0 answers
4 views

Item clustering to help with extremely sparse collaborative filtering

I have a large collaborative filtering dataset, where items are images. Some have text, other don't, some are quite visually similar. I do know that some items are similar, e.g. I have 10 distinct ...
user avatar
  • 199
1 vote
1 answer
36 views

Infer limits of unscaled values from their standardized values - Clustering

I am working on a clustering problem and I have some skewed variables. So, I log transform them and use them in clustering. However, instead of multivariate clustering, I do multiple univariate ...
user avatar
  • 1,872
1 vote
1 answer
48 views

Converting unsupervised to supervised problem - Overfitting - bad?

I am working on a customer segmentation using 5 features such as recency, frequency, monetary, tenure, unique_product_cnt etc. So, I did a RFM based segmentation where I used ...
user avatar
  • 1,872
0 votes
0 answers
27 views

Jenks optimization - goodness of variance fit interpretation

I am working on clustering/grouping 1D data. I am trying to find bins of multiple variables seperately. So, I tried the jenks natural breaks algorithm. Based on the ...
user avatar
  • 1,872
0 votes
0 answers
9 views

ML algorithm for distinguish between different activities

I have a simulated dataset which includes different activities like running, walking, eating etc now i want to apply ML algorithm to find how that algorithm can be used to distinguish between ...
user avatar
0 votes
1 answer
35 views

standardization/normalization for 1D clustering?

I have two input variables revenue and age. Am trying to find different bins within that variables. For ex: I have ...
user avatar
  • 1,872
0 votes
1 answer
34 views

Different distance matrixes result in the same clustering

I am using: the cluster::pam function in R for clustering distance matrix computed using Gower distance in cluster::daisy in R The issue I am having is that I run pam 2 times, with a different ...
user avatar
0 votes
0 answers
16 views

Appropriate to replace -inf with 0 after log transform? [duplicate]

I have a dataset of customers and their purchase data. Meaning, for each customer id, I have variables indicating number of unique products they bought, Number of online orders they placed, How many ...
user avatar
  • 1,872
0 votes
1 answer
56 views

silhouette score vs Distortion score

I am working on segmenting my customers with clustering. My dataset size is 7315 rows and 30 features. So, as a beginner to clustering, I passed all my 29 features (excluding id column) to the cluster....
user avatar
  • 1,872
9 votes
2 answers
628 views

What are the k-means algorithm assumptions?

I'm trying to understand what are the assumptions/hypothesis underlying the k-means clustering algorythm; specifically, I'm looking for a research/academic paper listing such hypothesis and explaining ...
user avatar
  • 325
1 vote
1 answer
23 views

Reliable methods to validate clustering of text phrases?

Question is in the title. I have clustered the word embeddings of text phrases, and now want to try and check whether the resulting clusters are coherent enough. I have tried methods that are ...
user avatar
2 votes
1 answer
47 views

Why not link features instead of selecting them - Clustering

Currently, I am working on customer segmentation using their purchase data. I plan to use clustering techniques. So, my data has below info for each customer (9 features and 1 id field) Now I am ...
user avatar
  • 1,872
1 vote
1 answer
30 views

Meaningful to retrieve original value after standardization using clustering

I already referred these posts here and here. Currently, I am working on customer segmentation using their purchase data. So, my data has below info for each customer Based on the above linked posts ...
user avatar
  • 1,872
0 votes
0 answers
22 views

ANOVA interaction effect

https://i.stack.imgur.com/wdXwB.jpg If I'm performing ANOVA using the formula in R: aov(Ileibacterium ~ diet*cage, data) Context - there are 3 cages per diet with ...
user avatar
  • 11
0 votes
0 answers
11 views

Are gaussian mixture models for clustering robust to data sparsity?

I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
user avatar
0 votes
0 answers
21 views

When taking a sample of a population with clusters, how do we set a threshold so that the clusters are significant?

For example, say we have a population of about 200,000 datapoints which can be sorted into about 10 clusters. However, performing the clustering process on the entire population is not feasible. Thus ...
user avatar
1 vote
1 answer
22 views

How would I separate a data set into a large number of pairs?

I have $n$ observations in $\mathbb{R}^2$ (where $n$ is large and even), representing points in physical space. What I am trying to do is to produce a set of $\frac{n}{2}$ pairs, such that the ...
user avatar
1 vote
0 answers
13 views

Using Variance of time series as input feature for time series clustering

I have a time series dataset, it is a data frame with 2000 rows and 1000 columns. Each rows is for one specific id and has a unique pattern. I want to clustering this data into multiple classes. Let ...
user avatar
  • 23
0 votes
1 answer
37 views

RFM Customer segmentation - Why Avg monetary value instead of total monetary value?

I am trying to segment our customers based on their purchase data. And I came to know about the RFM technique (Recency, Frequency and Monetary) through these posts here, here etc. Recency - How ...
user avatar
  • 1,872
2 votes
1 answer
93 views

Different Meanings of "Clusters" in Statistics

Typically, I have always come across the term "cluster" within statistics as reference to "clustering" - that is, for example the "K Means Clustering" algorithm. Recently,...
user avatar
  • 5,892

1
2 3 4 5
77