Questions tagged [clustering]

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Detecting anomalies in natural vibration data

I'm looking into using clustering-based method to detect anomalies in vibration signals. The idea is to extract features for every single sliding time-window of a time series of normal vibration data, ...
0
votes
0answers
9 views

How should I find similarities between individuals with multiple measures per individual?

I want to identify individual monkeys from a camera trap survey. From my accumulated camera trap videos, I see some variation in tail tuft size that I think could help me identify monkeys. I took ...
1
vote
0answers
25 views

How to create clusters in a one-dimensional space that would minimize the variance in the clusters?

We have numeric data in one-dimensional arrays where adjacency matters, e.g. [34,66,87,97,105,43,96] We want to cluster together those values, based on proximity, ...
0
votes
0answers
6 views

Property of a cluster in OPTICS clustering

Using OPTICS algorithm I organized a set of documents, based on their layout, into a hierarchy of clusters. Looking into top clusters, close to the hierarchy root, I can see they contain documents ...
0
votes
0answers
8 views

Clustering of behaviour-related data

I'm quite beginner in this field but now my research requires some methodology and I thought to create a topic, maybe somebody had the similar issue before. I have some data regarding to health-...
0
votes
1answer
20 views

Does it make sense to use feature selection methods to reduce dimensionality for unsupervised clustering?

If I have a dataset that is labeled with positive and negative examples, and I'd like to cluster (i.e. unsupervised) only the positive examples, does it make sense to reduce dimensionality using ...
0
votes
0answers
11 views

How to perform a classification model on clusters derived from cluster analysis

I'd like to compare whether classification models using a clustering technique before classification gives better predictions than classification models without clustering. Quite similar (but more ...
1
vote
0answers
5 views

Dimension reduction - doing a PCA on the coordinates of a MCA

I have a dataset with 25 continuous variables and 2 categorical variables. I want to perform k-means clustering, so as a previous step I am performing a multiple correspondence analysis on the ...
1
vote
1answer
29 views

Geometric significance of the dimensional reduction part of spectral clustering?

While performing spectral clustering of the original data $\{x_1,...x_n\}$, $ x_i\in \mathbb{R}^{d\times 1}$ (column vectors), into $k$ clusters, we Step 1: take the first (smallest) $k$ (column) ...
2
votes
0answers
9 views

How Do I Detect Outliers From Clustered Data Points?

I have a data set on a single variable (say, x). Below is the point plot of the data. From the plot it is seen that data points form few clusters around values, say 4, [1.5,2] and 0. Can we say that ...
1
vote
1answer
26 views

Does poor clustering results entail poor classification results? [closed]

In my project, I try to predict a class with 3 possible values. Before applying any classification algorithm I used clustering and noticed that for k=3, there is an equal distribution of those 3 ...
1
vote
0answers
9 views

Pooled panel regression with group-wise clustering by time

How can I compute t-statistics for the coefficients of the pooled panel regression model below such that I account for group-wise clustering by time? $$ Y_{it} = \alpha + \beta x_{it} + \epsilon_{it} ...
1
vote
1answer
37 views

Error bars in a population and subtracting two populations with different error bars

Suppose I want to measure a physical quantity. Let's say that $N$ trials were performed each with individual outcome $x_i,\quad i\in (1,N)$. Then obviously the outcome of the experiment would be the ...
0
votes
0answers
12 views

Interpretation of two way step clustering results [closed]

I use the classify > two step clusters In clusters vizual option I receive for a variable named variable1, variable2 into two cluster the following results ...
0
votes
0answers
8 views

Semi Supervised clustering

I have a dataset of points that belong to 10 different classes. These points are all unlabelled, besides one per class. In other words, I have a training set X of 10 points and a test set Y with N ...
0
votes
0answers
27 views

Determine modes in a histogram [closed]

If there are multiple distributions in a histogram(multimodal distribution), how can I find number of modes and cluster those modes? Current Approach Currently I'm using mean shift to find number of ...
0
votes
0answers
27 views

K-Means and HCA comparison to a model solution

I’m running several cluster analyses on related datasets and would like to find out which one is closest to the benchmarks/model solutions I would expect based on theory (or otherwise which benchmark ...
0
votes
0answers
32 views

Mean shift clustering and the curse of dimensionality

I've often come across resources that mention that mean shift based clustering doesn't work well in higher dimensions. The sources are as follows: Page 1 of https://www.ncbi.nlm.nih.gov/pmc/articles/...
0
votes
0answers
19 views

PCA/LDA interpretation and suitable clustering method

I am exploring the data on suicides from Kaggle: https://www.kaggle.com/russellyates88/suicide-rates-overview-1985-to-2016 I wanted to perform the dimensionality reduction to identify possible ...
2
votes
1answer
24 views

Boundary estimation using statistical techniques

Do you know a good methodology to estimate the boundary between two sets? Here are the specifics of the problem: I am studying a recursion defined as $$ 2(n+q) x_{n+2}= (r(n+q) +s)x_{n+1}+((2-r)(n+q)...
0
votes
0answers
20 views

Is there any proof that clustering algorithm reduces forecasting error?

k-means clustering algorithm is very useful for forecasting a future value (in a sense of time-series) by allowing estimation only within a chosen cluster. I can intuitively admit that k-means ...
1
vote
0answers
19 views

Number of clusters exclusive to hierarchy

I'm working on hierarchical clustering and I stumble on the "where to cut the tree" question. I know there are some methods that can suggest the optimal number of clusters like elbow, gap, silhouette ...
3
votes
1answer
43 views

Dynamic Time Warping Univariate Time Series to aid in selecting Forecasting Model

I have approximately 174 univariate time series that I would like to forecast. These are all country observations that have been thoroughly cleaned with no outliers or missing values. I would like to ...
1
vote
0answers
23 views

Identifying original elements in a permutation of a pattern

In the pattern above, the blue peaks are known and are labeled $p_1, p_2, ... p_N$. The red line is the same pattern of peaks, but altered slightly. Given a blue pattern and red pattern, I want to be ...
0
votes
0answers
9 views

How to overcome the issue of singular within-cluster scatter matrices in clustering using entropy-based feature ranking?

I am trying to implement the entropy-based feature selection method for clustering by Dash and Liu. In this method, features are ranked in importance based on an entropy minimisation procedure and ...
1
vote
1answer
40 views

One sample t-test with Groups?

I wish to find the mean dollar value in my data while properly accounting for different groups. Say for example I have a set of data as follows: ...
-2
votes
2answers
25 views

Clustering categorial data

I have a dataset with 100 nominal categorical variables with two levels each, for example: Do you smoke? Yes/No Do you like to dance? Yes/No etc. How can I cluster this dataset for try to group/...
0
votes
0answers
14 views

Cluster Events of specific time window

i specify a timewindow of some minutes and extract occurences of certains events. The resulting matrix will look like this: ...
0
votes
0answers
15 views

Heuristics for approximating the epsilon parameter for the DBSCAN algorithm

The classic heuristic for approximating the epsilon parameter for DBSCAN consists of manually visualizing a k-nn distances plot as proposed by the original authors of the algorithm [1]. I want to use ...
2
votes
0answers
33 views

Clustering with ranking data [duplicate]

I'm just trying to get some practice with implementing some Machine Learning algorithms with a variety of data. I'm just getting started so I'm pretty new to the whole process and would appreciate any ...
-1
votes
0answers
18 views

Community Detection exercice

I am developing a list of high dimensional statistics exercises and I came across the following exercise: I have trouble understanding the structure of matrix $\mathbf{B}^*$ and finding the sums ...
0
votes
1answer
25 views

Distinighsing Between Groups on a Bimodal Varaible

I am working with the diamonds data set from the tidyverse package in R. library(tidyverse) View(diamonds) When I plot a histogram of the price variable with 300 ...
0
votes
0answers
16 views

How does clustering will affect the MAPE and RMSE?

MAPE and RMSE are two very popular techniques to calculate the error. Now assume I have time series and cluster them to K clusters. This might reduce the training time when we are using the ...
1
vote
0answers
12 views

Clustering evaluation and adjusted rand index

How can we evaluate clustering methods in a efficient way? Imagine I generate artificial data and the true clusters are something like "A,A,A,A,..,A,B,B,B,..,B". After I applied some clustering method ...
0
votes
0answers
23 views

Cutoff values for Dunn index?

I'm trying to find cutoff values for the Dunn index to assess clusters, but it has not been possible for me. I found cutoff values for the silhouette https://www.stat.berkeley.edu/~s133/Cluster2a....
0
votes
1answer
24 views

K-means to cluster texts, scaling

I want to cluster a folder of texts. I created a data file where for each text, I write whether a certain word appears in it or not. I want to cluster according to this. So my matrix is globally only ...
0
votes
0answers
15 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
0
votes
1answer
21 views

Are there any clustering techniques that work well on galaxy arm dataset?

Are there any clustering techniques (k-means, GMM) that work well for this dataset?
0
votes
0answers
17 views

t-SNE Clustering

I have about 500 users and their travel behavior as 100-dimensional vectors, created with a doc2vec approach. Using tensorboard´s embedding projector I can visualize these in a 3 or 2 dimensional ...
0
votes
0answers
18 views

Internal clustering criteria as a measure to know if feature transformation is useful

Does it make sense to use internal clustering criteria, such as Calinski-Harabasz, Davies-Bouldin and Silhouette, to draw conclusions about whether feature scaling makes sense with the existing data? ...
0
votes
0answers
9 views

Signal/Wavelet Clustering

Problem Setting In an experiment: I have 3 signal sources and 10 sensors each generating wavelets as time goes by. The distances from each source to sensors change from experiment to experiment. ...
3
votes
1answer
46 views

Why are K-means and GMM (Gaussian Mixture Models) not suitable for discovering clusters with non-convexs shapes?

I have seen that mainly here and from a lot of resources that K-means and Hello all! Gaussian mixtures are not suitable for detecting clusters with non-convex shapes. I know that because both ...
1
vote
1answer
35 views

Clustering Proof of Equation

Greetings! Could anyone enlighten me about the validity of this equation? I'm trying to prove it without success. $K$ is the number of clusters, $C_i$ is the $i$-th cluster, $m_i$ is the number of ...
1
vote
1answer
17 views

Relationship between Cohesion and Separation

Hello! May someone explain to me please how to prove the equation depiicted with the red line? Thank you in advance!
2
votes
1answer
40 views

Assign M cars to N parking lots with limited spots

I have a dataset with $M$ static cars locations and an information about $N$ parking areas with $k$ slots there. What clustering algorithm can I use to assign all cars to the parking zones in ...
0
votes
1answer
12 views

What's the formal term for the “Group of points that have X as their neighbor”?

Due to asymetrical nature of K-NN, the points neighboring X need not be the same as the points which have X as their neighbor. Is there a formal term to designate those points which have X as one of ...
-1
votes
1answer
15 views

Recommender Engine for documents VS Search engine indexing

I have a lot of books and I want to make recommendations to users based on the description and the title of those books. I think that one way is to preprocess the content of the title and ...
1
vote
1answer
25 views

No clear elbow and low Silhouette scores K-Means

I am implementing the K-Means algorithm to group books based on their title and their description. I pre-processed the data merging the fields and deleting all the punctuations and some undesirables ...
1
vote
1answer
20 views

How to account for similarities in features when clustering?

I have survey data from students who were asked to rate (yes/no) if a math equation was useful for a set of four educational purposes. Students were ask to answer this survey for five different math ...
-1
votes
1answer
12 views

How to unsupervised-cluster of binary vectors?

I have a set of binary vectors of roughly 500 dimensions. For EDA purposes mainly, I'd like to cluster them, maybe hierarchically. What could be the right distance metric for my problem? Is the ...