Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

learn more… | top users | synonyms (1)

6
votes
0answers
75 views

Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
6
votes
0answers
355 views

Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
6
votes
0answers
301 views

Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal ...
5
votes
0answers
142 views

Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
4
votes
0answers
137 views

Multiple eigenvectors in graph spectral clustering

In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by eigenvector of a ...
4
votes
0answers
76 views

What is the difference between affinity matrix eigenvectors and graph Laplacian eigenvectors in the context of spectral clustering?

In spectral clustering, it's standard practice to solve the eigenvector problem $$L v = \lambda v$$ where $L$ is the graph Laplacian, $v$ is the eigenvector related to eigenvalue $\lambda$. My ...
4
votes
0answers
285 views

Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
4
votes
0answers
1k views

Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
4
votes
0answers
128 views

Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
4
votes
0answers
186 views

Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
4
votes
0answers
76 views

How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
4
votes
0answers
646 views

test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
4
votes
0answers
215 views

What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
4
votes
0answers
90 views

R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
4
votes
0answers
240 views

Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
4
votes
0answers
414 views

How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
4
votes
0answers
80 views

In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
3
votes
0answers
31 views

Help in understanding a clustering technique using neural network

I am having difficulty in understanding a technique for clustering and segmentation of biomedical images using the concept of time series. The paper on which the Question is based is : M. Lacomi et. ...
3
votes
0answers
34 views

Topology of Confidence Intervals

I hope this is the right site to post this. The example I have in my mind is a GLMM model, where we infer random effects, and a random effect caterpillar plot (with confidence intervals): Now, ...
3
votes
0answers
40 views

Can SVD be used to perform factor analyis?

What is the relationship between SVD and factor analysis? How can use singular values and other matrices from SVD to perform factor analysis or cluster document-term matrix without using other ...
3
votes
0answers
33 views

A high cophenetic correlation coefficient but dendrogram seems bad

I have 2 results for the same dataset. One is hierarchical clustering using Ward's method and I got 0.75 cophenetic correlation coefficient. The second is average method and I got 0.91 cophenetic ...
3
votes
0answers
44 views

Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
3
votes
0answers
70 views

How to cluster/analyze effect sizes after meta-analysis? (meta-meta-analysis)

For a research project I compared persons with and without a specific disorder on basically every published outcome I could find. The idea was to get some sort of profile of this disorder (i.e. skills ...
3
votes
0answers
51 views

Theoretical link between the graph diffusion/heat kernel and spectral clustering

The graph diffusion kernel of a Graph is the exponential of its Laplacian $\exp(-\beta L)$ (or a similar expression depending on how you define the kernel). If you have labels on some vertices, you ...
3
votes
0answers
54 views

Reconciling wildly different $\chi^2$ independence test results (bootstrap clustering)

I have some data that should be randomly assigned to treatment $T$, and am running some tests on observables to give evidence that this is indeed the case. Let's focus on an outcome I'll call $X$, ...
3
votes
0answers
119 views

Highly Connected Subgraphs cluster

i'm studying a method to cluster similar topic represented in a graph like this: The result must be: [0] = 1,2,4 [1] = 3 I tried Markov Cluster Algorithm but ...
3
votes
0answers
93 views

How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...
3
votes
0answers
130 views

Use of Hidden Markov Models for Clustering

I would like to ask whether Hidden Markov Models can be used for clustering and if so, in what cases. I have found somewhere, references like this but practically I haven't found a way to do this. Is ...
3
votes
0answers
41 views

clustering for histogram shapes

I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data: number of bins vary from sample to sample (larger price range ...
3
votes
0answers
86 views

Regarding the size of training data for building classifier

When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...
3
votes
0answers
604 views

Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
3
votes
0answers
1k views

Problem with R code for spectral clustering

am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script ...
2
votes
0answers
33 views

Spectral clustering or hierarchical clustering for this senario?

I have a data set of about 40,000 time series. The length of each time series is 64. I consider these 64 as features for the data. I want to cluster data into groups which have similar time series (I ...
2
votes
0answers
62 views

Matching with multilevel data

I've got a dataset where a treatment $W$ has been applied to units $i$ within clusters $c$. $W$ is constant within each cluster. As a component of an algoritm that I'm implementing (which was ...
2
votes
0answers
22 views

Why does original paper introducing DBSCAN state it takes only one input parameter?

I am reading about the DBSCAN clustering algorithm from the original paper by Ester et al. In this paper, the authors state in the abstract, the introduction, and the conclusion that their algorithm ...
2
votes
0answers
66 views

Separation of points clouds via classification methods

I have multiple images from a 3D-Scanner in point cloud form. Part of the image is a fixture to hold the object to be scanned. I want to extract the object itself by classifying the fixture and the ...
2
votes
0answers
36 views

Appropriate distance measure

I'm working with a dataset containing 600 matrices (dim: 44x3) with numerical results of applied tests (pass, fail, toVerify). Data is organized in rows. Row nr 1 represents test nr 1. Tests can be ...
2
votes
0answers
53 views

Best practices for reducing/clustering data which is location/time based?

I have event-based data (or longitudinal data) which has a person's age, gender, location (zipcode etc.), the datetime when they saw something, and how long they watched it. I have a project ...
2
votes
0answers
50 views

Power calculation for multiple level clustering / randomization using simulations R

I am trying to calculate power of the following design: -Treatment is randomized over a small number of clusters (1st level clustering=regions) -Within each region we randomly select villages ...
2
votes
0answers
96 views

Choosing $k$ in consensus cluster plus using cophenetic correlation coefficient

I am trying choose best $k$ from the consensus clustering using the Cophenetic Correlation Coefficient (CCC). I tried as follows. The correlation coefficients values are poor, i.e., ...
2
votes
0answers
405 views

R: silhouette with k-means

I'm currently checking some clustering evaluation indexes in R, and now I'm using Silhouette and its respective function in R, "silhouette" (in "cluster" package). To test the method, I used the ...
2
votes
0answers
220 views

converting continous variable to discrete while maximizing gini

I have a continuous independent variable which is used to explain the dependent binary variable in logistic regression. The model user's requirement is to group this continuous variable to 6 bins. I ...
2
votes
0answers
28 views

Profiling survey respondents with both categorical and ordinal variables

I have a mixture of categorical and ordinal variables from a survey that I am trying to use to create "profiles" or segments that differ from one another with respect to a dependent variable (the ...
2
votes
0answers
102 views

Spatial clustering based on response

Statistics version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: f(a,b,c;x,y), with f being a vector of several quantities. I would like to ...
2
votes
0answers
31 views

watershedding vs mean shift clustering

I'm working on a clustering problem, and I was trying to understand the difference between the watershed algorithm and mean shift clustering. It seems that these algorithms are popular for image ...
2
votes
0answers
22 views

Clustering stream of new customers based on future potential

I need to cluster new customer according to their future potential, but I have only information about their first transaction. I have access to all transactions for the other customers. So I can do ...
2
votes
0answers
103 views

Variational Inference: good inference but ELBO decreases instead of increasing

I am playing with Variational Inference for clustering within a mixture of Gaussians. My first implementation seems to work fine (this is for the geyser dataset): ...
2
votes
0answers
43 views

Observations get in a line in a PCA score plot. Something wrong with the data?

I ran a clustering and in the resultant PCA score plot some observations getting in a line drew my attention (I marked them with a red line) . How come they distribute like that? I doubt there is ...
2
votes
0answers
73 views

How do you do EM algorithm for a factored model for a recommender system?

Let $X$ be a $n \times d$ matrix with users as rows and movies as columns. Each user is a single row $x^{(u)} \in \mathbb{R}^d$ (i.e. for user u there are at most d ratings for the d movies). Also ...
2
votes
0answers
231 views

jenks natural breaks vs k-means

I am new to this topic. As far as I know both are data clustering methods. Then my question is when is Jenks prefered over k-means? I read on this website that jenks is particularly suited for ...