Questions tagged [clustering]
Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]
3,988
questions
0
votes
0
answers
3
views
Should main effects and interaction terms be included in Kmeans clustering? (hierarchical principle in clustering analysis)
Let's say I'm trying to cluster observations based on five features, including:
n_emps: Number of employees
n_cust: Number of ...
- 443
0
votes
1
answer
26
views
Elbow method for tuning DBSCAN when the minimum number of points per cluster is one
The elbow method calls for setting the number of nearest neighbors (let's call it $k$) to the minimum number of points for a cluster (let's call it $m$), but what do you do when $m\leftarrow1$? Is the ...
0
votes
0
answers
27
views
Clustering algorithm recommendations
So I have somewhat of a clustering problem. What you see in the image is a set of points that need to be classified or clustered in six rows. These particular points have already been classified, that ...
0
votes
0
answers
32
views
SVM kernels corresponding to different types of distance measures
This answer to Data normalization for RBF kernel points out that RBF kernel implies Eucledean distance. Are there kernels corresponding to other popular distance/dissimilarity measures, such as Bray-...
- 3,522
0
votes
0
answers
10
views
Clustering of Variables around Latent Variables (CLV) over only qualitative data
I'm reaching out today because I have a concern regarding the clustering approach employed with the CLV method introduced by Vigneau and Qannari in 2003. I've noticed that this method is predominantly ...
- 11
0
votes
0
answers
15
views
Expectation Maximization on Multivariate Gaussian Mixture Model for clustering
I have a dataset with 1000 observations and two features that define those N=1000 data points. Hence it is 1000*2 input matrix. I need to cluster them into k clusters.
I am not understanding the E-M ...
- 1
0
votes
0
answers
14
views
Mixture Model: Data Consist of Continuous and Binary Features
I have a features like below
id
x1
x2
x3
x4
x5
id1
0.4
1.4
5.6
1
0
id2
-0.01
0.5
-3.4
0
1
where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
0
votes
0
answers
21
views
Weighted regression with no variance within cluster
Given a clustered data set with no variation within a cluster, shouldn't a regression weighted with the inverse cluster size give the same results as a regression with only one observation per cluster?...
- 170
0
votes
0
answers
13
views
Identify regression groups
I have time series data with some value and corresponding time. These measurements are divided into groups (I plotted each group in a separate plot). I want to build a linear regression model for each ...
- 35
0
votes
0
answers
22
views
Turning heatmap into clusters - Classification
Assume that you having a heatmap that looks like this. The goal is to classify all the "dot" inside the image. How can that be done?
The assumptions of the image:
The image has always black ...
- 349
-1
votes
0
answers
19
views
Please help me evaluate these results if a Bayesian Mixture Model is better than K-Means Clustering
Dataset
I am performing Clustering in this dataset which some samples are:
Now, I am comparing the results of:
K-Means Clustering
Bayesian Mixture Model (BMM)
I set $K=6$ clusters for both of them ...
- 601
0
votes
1
answer
24
views
Comparison of time series: Cluster behaviors / detect anomalies
I am studying a dataset of time series for different users. The dataset contains records of actions (or registrations) of the users over time. I have data of a whole week for about 80,000 users.
...
- 1
0
votes
0
answers
19
views
Can I use clustering to compare diseases, after partial least squares correlation?
I have two datasets from patients with different diseases, which contain multiple measures of regional brain function and performance on different memory tasks. The memory tasks used in each dataset ...
- 121
1
vote
1
answer
20
views
Is there a way to automatically split large clusters that are greater than some maximum number of points?
I ran HDBSCAN on these coordinates and got some clusters but some are too large. HDBSCAN has a minimum cluster size parameter, but no maximum size. All I want is to intuitively divide larger clusters ...
- 11
0
votes
1
answer
23
views
Normalize vs Standarization with percentage data
I'm studying access patterns to a facility with clustering.
My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
- 219
2
votes
1
answer
59
views
Constrained, Supervised Classification
I am playing around with a classification model, and I would like to know if there are any known methods to achieve what I am looking to do.
The data looks something like this:
Class
id
$x_1$
$x_2$
.....
- 75
1
vote
0
answers
31
views
Which averages in a set are probably identical
Suppose there are $N$ types of blocks. The blocks are labelled with their types. The average weights of the types are in the set $\{w_i\,|\,\,i<N\}$. In other words, at least one pair of block ...
- 123
0
votes
1
answer
11
views
Deriving a boundary for a single cluster data
Summary
I have a dataset of 2D points that exhibit a distinct pattern. My goal is to create a boundary that encompasses the main cluster of points while excluding outliers. In other words, if a point ...
0
votes
0
answers
9
views
Criteria for a Distance Metric to be Compatible with K-means Clustering
Referring to this post, it's mentioned that K-means clustering doesn't inherently rely on the pairwise distances between data points, and not every distance metric is suitable for k-means clustering.
...
- 299
0
votes
0
answers
23
views
Cluster Validation on Hamming Distances/K-Modes
I wondered if anyone could recommend internal cluster validation resources in R capable of estimating $k^*$ prior to using k-modes?
Therefore, I wondered if anyone knew of existing internal validation ...
- 244
0
votes
0
answers
3
views
Criterion to assign individuals to clusters in bayesian mixed model with distribution of probabilities
I have a dataset with a set of individuals indexed by $i = \{ 1, ..., N \}$, and I make a number of measuremenets under two conditions for each individual to measure the effect $\beta$ of my ...
- 448
0
votes
0
answers
10
views
Hierarchial clustering given a list of banking syndicates
I have a list of banking syndicates (groups of banks), and I'm trying to do some hierarchical clustering on it to gather if there is some connection between them (e.g. if lots of syndicates share the ...
- 121
1
vote
1
answer
53
views
Why gaussian mixture model is sensitive to cluster size?
I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
- 123
0
votes
0
answers
20
views
Finding the best number of clusters including only one cluster (so no clustering)
Suppose you have a number of small pieces of signal from a sensor. These snippets are found earlier using signal detection. These snippets can be real events or noise. I now want to cluster these ...
- 121
1
vote
0
answers
15
views
Counts, percentages and scaling in clustering
I am working in a clustering.
I have a dataset with the accesses of a series of subjects to a facility (each row register the subject ID and the date/time of the access)
I can, for example, count the ...
- 219
0
votes
1
answer
45
views
How to use VAR model for model forecasting
Good evening, everyone.
I am a software engineer and I am studying the VAR model and its advantages and disadvantages. Specifically, my question is the following: is it possible to use VAR to predict ...
0
votes
1
answer
19
views
Why do we always consider 2-variable affinity metrics?
Clustering algorithms often use some affinity metrics to cluster a dataset. Given some data points $x_1, x_2, \ldots, x_n$, it is common to compare $x_i$ and $x_j$ using a function $f(x_i, x_j)$ which ...
- 1
0
votes
0
answers
14
views
Would it be wise to feed k-means results into a cmeans?
I am conducting a cluster analysis of documents using document embeddings as the input to the algorithms. One of the problems I am coming across is that in reality there are documents that belong to ...
- 31
0
votes
0
answers
34
views
Calinski-Harabasz score as a validity index for hierarchical clustering with distance matrix
I have a distance matrix that I want to performed hierarchical clustering with, and find the optimal number of clusters by maximizing the Calinski-Harabasz (CH) score. However, I only know how to ...
0
votes
0
answers
13
views
Good distance metric for clustering frequency variables?
I have a dataset of customers, with variables refering to ABSOLUTE FREQUENCIES (counts). For example, I have the number of times he makes a purchase in weekend, the number of purchases in laborable ...
- 219
0
votes
0
answers
11
views
Rank clusters based on mean feature value and variance feature value of the cluster
I would like to rank clusters according to their importance.
From 113 demand time series data I extract the following 6 features from each time series.
"Average Demand", "n95 demand&...
- 244
1
vote
1
answer
28
views
How to tell whether segments from K Means clustering result are "successful" and will impact business metrics?
Background
I'm a data analyst. The Business unit I'm assigned for needs to segment users based on power vs non-power users so they can target each segment with proper treatments.
Goal
Segment users (...
- 135
1
vote
1
answer
49
views
Approximate Gower's dissimilarity measure
I have a very large dataset with mixed-type variables. When I apply the Gower's dissimilarity measure to obtain the distance matrix, it is running out of memory. Due to the large size of the data, it'...
- 41
0
votes
0
answers
23
views
Graph clustering algorithm for two clusters of same size
I have a graph of 100 nodes. I know that there are 2 clusters of nodes of equal size 50. I created (kind of like a training set) the following instance of the problem (see below, I have some values to ...
0
votes
0
answers
10
views
number of parameters in Dirichlet Mixture Model clustering (non-bayesian)
I made a function that implements the clustering algorithm in the research article "Clustering compositional data using Dirichlet mixture model" (2022). I am now trying to figure out which ...
0
votes
0
answers
16
views
Grouping continuous data with a skewed shape
I am currently working with a dataset of Facebook posts that have variables such as total engagement and views. I am trying to group these two variables into discrete levels according to how big the ...
- 1
0
votes
1
answer
33
views
Dummy Variable Trap in KMeans Clustering
My data set is having a column Gender, so I have to apply One Hot Encodingto perform KMeans Clustering.
Q1. Should I take care about ...
1
vote
0
answers
53
views
Clustering or factor analysis for dimensionality reduction in multivariate linear regression
I have dataset describing aggregated purchases from multiple brands. It contains variables:
Brand (ordinal)
Promotion (ordinal)
Sales (numeric)
I need to use linear regression to describe the effect ...
0
votes
0
answers
50
views
When using a gower dissimilarity matrix for use in dbscan clustering, the bools still perfectly separate the clusters
I am working on a clustering analysis where my data is a mix of bool and numeric. From research it looks like one should not do a simple kmeans with mixed data types since the extreme end of scaled ...
- 1,558
2
votes
1
answer
45
views
Inconsistent cluster indices using hierarchical clustering for time series data
I am currently trying to spatially cluster data that is ordered on a grid. Each point has x and y coordinates as well as a measurement value. These features come from a time series where I analyze ...
- 137
0
votes
0
answers
37
views
How to deal with the high dimensionality when using EM algorithm to solve Gaussian mixture models?
When I use the EM algorithm to solve a Gaussian mixture model, we may encounter the computation of Gaussian densities in the E step. Specifically, we should have the posterior probability as
$$
\pi_{...
- 1
0
votes
0
answers
35
views
How to do 1D interval clustering into clusters with equal number of intervals?
I am trying to analyse genomic data. They are organized in the form of records.
A record looks like the following:
...
- 416
0
votes
0
answers
20
views
How to deal with a few number of cluster (n<15) for nonlinear?
I am currently trying to address a few number of clusters (n<15) for a nonlinear model (multinomial logit).
I have already gotten an idea from Cameron et al. (2008) for a linear model to use "...
- 61
0
votes
0
answers
16
views
Can I use K-means clustering labels as my dependant variable for logistic or HB analysis?
Background
I am investigating Best-Worst Scaling analysis methods.
For those not familiar a survey is presented showing some attributes and the respondent is asked to score 2 of the shown attributes, ...
- 1
0
votes
1
answer
51
views
Using Word Embeddings in Clustering and Topic Modelling
I am new to the field of NLP and would appreciate any guidance please. I am trying to understand how word embeddings can be used in clustering and topic modelling. If I create word embeddings for ...
- 31
2
votes
1
answer
30
views
Comparing partitions with different number of clusters
Say that we have a ground-truth partiton $U$ consisting of $k$ clusters and two partitions $V_1$ and $V_2$ obtained via two different algorithms :
$V_1$ consists of $k$ clusters.
$V_2$ was obtained ...
- 63
0
votes
0
answers
40
views
Is "ward.D" a good agglomeration method in R?
I need to do clustering on a large scale file (~12M rows, 18 features + id index). As a first step, i tried different algorythms in Python with a test sample (40k rows) which gave results (clearly ...
0
votes
0
answers
14
views
Clustering algorithm for identifying circuits in time-stamped geolocation data
Context + Overall Aim:
We have months of data tracking where a truck has travelled around a mine site, and we need to identify circuits it travels. The purpose of the truck is to essentially transport ...
- 1
0
votes
0
answers
14
views
Cluster based on random effects, STAN
I have a problem where I measure repeated responses in condition A and in condition B for a set of individuals $i=1,...,n$. I am interested in learning about the effect of the condition in the ...
- 448
0
votes
0
answers
7
views
Distance metric to compare statistical features extracted from 400 time series for time series clustering
I would like to cluster 400 car rental demand time series (small positive valued) based on the following 7 statistical features:
entropy, number of mean crossings, 95th percentile, root mean squared,
...
- 244