# Tagged Questions

Cluster analysis is the task of partitioning data into subsets of objects according to their mutual "similarity," without using preexisting knowledge such as class labels. [Clustered-standard-errors and/or cluster-samples should be tagged as such; do NOT use the "clustering" tag for them.]

77 views

### Territories from observations

I have a number of animal observations, and want to deduce the number of territories (i.e. the number of individual animals) from this. More formally, the problem can be stated as follows: Each ...
367 views

### Clustering & Time Series

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset. Now I want to see ...
309 views

### Recommended method for finding archetypes or clusters

I wish to cluster users together in a database, with each user represented by a number of features that are both discrete and continuous in nature. The aim is to define a small number of archetypal "...
157 views

### Multiple eigenvectors in graph spectral clustering

In Newman's PNAS 2006 paper Modularity and community structure in networks, the first eigenvector splits the graph in two clusters, and then each cluster can be further divided by eigenvector of a ...
157 views

### Expectation Maximization Clarification

I found very helpful tutorial regarding EM algorithm. The example and the picture from the tutorial is simply brilliant. Related question about calculating probabilities how does expectation ...
94 views

### Cluster analysis vs Factor analysis as a means for “grouping” variables or cases: the differences

I've noticed responses that at face value seem to be in contradiction with each other. For instance, here @peter-flom writes Short answer: Cluster analysis is about grouping subjects (e.g. ...
97 views

### What is the difference between affinity matrix eigenvectors and graph Laplacian eigenvectors in the context of spectral clustering?

In spectral clustering, it's standard practice to solve the eigenvector problem $$L v = \lambda v$$ where $L$ is the graph Laplacian, $v$ is the eigenvector related to eigenvalue $\lambda$. My ...
286 views

### Detecting statistically significant clustering of continuous values

I'm working with biological sequence data where each position in the sequence has an associated continuous value. I'm ignoring the sequence content so the data is very similar to a time series with ...
1k views

### Sorting/Clustering similarity matrices

I wonder, what are the available libraries in R or Python to do correlation matrix clustering (sometimes it is referred to clustering). I also, wonder, after clustering/grouping each point. What is ...
138 views

### Which variables are driving correlations within groups

I'm running an analysis on a few data sets that each typically have 100-200 cases measured across 120-160 variables - something similar to looking at gene expressions. Each variable is a non-centered ...
199 views

### Cluster on high dimensional categorical data (Images with keywords)

We're looking for clues to perform a Cluster Analysis in a DB with +400K images which have keywords associated to them. Each image could have from 1 to 30 keywords. Total keywords count is +35K. ...
77 views

### How to randomize the tips of a functional trait dendrogram?

I have generated a functional trait dendrogram using species x trait and plots x species matrices through dbFD in package FD. I want to randomize the tips of the ...
672 views

### test the significance of clusters

Good morning, I am analyzing a dataset composed by 364 subjects and 13 binary variables (0,1 = absence,presence). I am testing possible association (co-presence) of my variables. To do this, I was ...
218 views

### What are pitfalls of bootstrapping on random sample of master data?

Will I obtain seriously biased results if I use bootstrapping on a subsample of a larger dataset? Rather than drawing 100 bootstrap samples from a dataset of 50 million + records, which could hog ...
91 views

### R: looking for “time” clusters in a data set

I am new to R and seeking some advice. I have a set (~20M) of data describing on which step a process did fail or succeeded: ...
250 views

### Category selection for text classification

It is said that to achieve a good result (many different metrics) for text classification, it is not always a business of selecting the algorithm/classifier. Sometimes, it is even more important to ...
434 views

### How do I adjust standard errors in a research study in which the control group is constructed via matching with replacement?

I have a treatment sample of 200 firms. I'm using propensity score matching to pair each treatment observation with one control (sampling with replacement, in order to minimize bias associated with ...
81 views

### In which case does FCM membership converge to 1/K?

I have tested the fuzzy C-means (FCM) algorithm using the R function fanny from the cluster package and I have wrote my own FCM ...
10 views

### Distance for Clusterization: Similarity Measure via Covariance Structure

I work with time series - vectors of length of $10^6$ real numbers. I have a lot of these vectors and use some algorithms that have $O(n^2)$ time complexity (n - number of samples), so if I will try ...
32 views

### Help in understanding a clustering technique using neural network

I am having difficulty in understanding a technique for clustering and segmentation of biomedical images using the concept of time series. The paper on which the Question is based is : M. Lacomi et. ...
38 views

### Topology of Confidence Intervals

I hope this is the right site to post this. The example I have in my mind is a GLMM model, where we infer random effects, and a random effect caterpillar plot (with confidence intervals): Now, ...
45 views

### Can SVD be used to perform factor analyis?

What is the relationship between SVD and factor analysis? How can use singular values and other matrices from SVD to perform factor analysis or cluster document-term matrix without using other ...
59 views

### A high cophenetic correlation coefficient but dendrogram seems bad

I have 2 results for the same dataset. One is hierarchical clustering using Ward's method and I got 0.75 cophenetic correlation coefficient. The second is average method and I got 0.91 cophenetic ...
65 views

### Is dimensional reduction using Autoencoders possible with a small sample size?

I have a data set that is not too big but high dimensional, let say 10000 dimensional. I want to use an autoencoder to extract relevant features (clusters) in the data. Usually when I have seen ...
83 views

### How to cluster/analyze effect sizes after meta-analysis? (meta-meta-analysis)

For a research project I compared persons with and without a specific disorder on basically every published outcome I could find. The idea was to get some sort of profile of this disorder (i.e. skills ...
57 views

### Theoretical link between the graph diffusion/heat kernel and spectral clustering

The graph diffusion kernel of a Graph is the exponential of its Laplacian $\exp(-\beta L)$ (or a similar expression depending on how you define the kernel). If you have labels on some vertices, you ...
54 views

### Reconciling wildly different $\chi^2$ independence test results (bootstrap clustering)

I have some data that should be randomly assigned to treatment $T$, and am running some tests on observables to give evidence that this is indeed the case. Let's focus on an outcome I'll call $X$, ...
145 views

### Highly Connected Subgraphs cluster

i'm studying a method to cluster similar topic represented in a graph like this: The result must be: [0] = 1,2,4 [1] = 3 I tried Markov Cluster Algorithm but ...
99 views

### How to build “supervised clustering” for neural networks?

I'm confused as to what the output would be. Consider the "blind source separation" problem. Let's say I have a ton of training examples where the input is the final cacophony of sounds as a sound ...
137 views

### Variational Inference: good inference but ELBO decreases instead of increasing

I am playing with Variational Inference for clustering within a mixture of Gaussians. My first implementation seems to work fine (this is for the geyser dataset): ...
147 views

### Use of Hidden Markov Models for Clustering

I would like to ask whether Hidden Markov Models can be used for clustering and if so, in what cases. I have found somewhere, references like this but practically I haven't found a way to do this. Is ...
45 views

### clustering for histogram shapes

I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data: number of bins vary from sample to sample (larger price range ...
88 views

### Regarding the size of training data for building classifier

When we build a classifier, like SVM or Naive Bayesian, are there any generic rules or theoretical derivations on the size of training data set? For example, to train a SVM-based classifier, what ...
631 views

### Cluster similarity percentages with inverted Y-axis in R

I'd like to ask a question here that I've also asked on Biostar (stackexchange) and someone there forwarded me to this website. I was wondering how I could perform a Bray Curtis similarity clustering ...
1k views

### Problem with R code for spectral clustering

am writing a simple R script to test the spectral clustering algorithm but for the eigenvalues I don't get them all positive and lambda0 is different from 0. here is my script ...
20 views

### Cluster analysis with PC scores (Multivariate Morphometrics)

I have a morphometric dataset of 430 fish, with 27 various "shape" measurements, for example: total length, head length, fin distances, etc. My goal is multi-fold: (BIG GOAL: cluster by "shape") ...
35 views

### Choosing evaluation measure for non-parametric clustering

I have to cluster some data using non-parametric clustering technique which is given in this paper. After all the cluster evaluation measure used in this paper is Normalized Mutual Information as they ...
51 views

### Spectral clustering or hierarchical clustering for this senario?

I have a data set of about 40,000 time series. The length of each time series is 64. I consider these 64 as features for the data. I want to cluster data into groups which have similar time series (I ...
58 views

### Using entropy to imputing missing value based on grey relational analysis and clustering

This algorithm contain three techniques : 1-fuzzy c-mean clustering 2-Grey relational theory 3-Entropy multiple imputation The frame work of this algorithm is as follows : My questions are ...
68 views

### Matching with multilevel data

I've got a dataset where a treatment $W$ has been applied to units $i$ within clusters $c$. $W$ is constant within each cluster. As a component of an algoritm that I'm implementing (which was ...
25 views

### Why does original paper introducing DBSCAN state it takes only one input parameter?

I am reading about the DBSCAN clustering algorithm from the original paper by Ester et al. In this paper, the authors state in the abstract, the introduction, and the conclusion that their algorithm ...
81 views

### Separation of points clouds via classification methods

I have multiple images from a 3D-Scanner in point cloud form. Part of the image is a fixture to hold the object to be scanned. I want to extract the object itself by classifying the fixture and the ...
38 views

### Appropriate distance measure

I'm working with a dataset containing 600 matrices (dim: 44x3) with numerical results of applied tests (pass, fail, toVerify). Data is organized in rows. Row nr 1 represents test nr 1. Tests can be ...
55 views

### Best practices for reducing/clustering data which is location/time based?

I have event-based data (or longitudinal data) which has a person's age, gender, location (zipcode etc.), the datetime when they saw something, and how long they watched it. I have a project ...
66 views

### Power calculation for multiple level clustering / randomization using simulations R

I am trying to calculate power of the following design: -Treatment is randomized over a small number of clusters (1st level clustering=regions) -Within each region we randomly select villages (...
130 views

### Choosing $k$ in consensus cluster plus using cophenetic correlation coefficient

I am trying choose best $k$ from the consensus clustering using the Cophenetic Correlation Coefficient (CCC). I tried as follows. The correlation coefficients values are poor, i.e., ...
682 views

### R: silhouette with k-means

I'm currently checking some clustering evaluation indexes in R, and now I'm using Silhouette and its respective function in R, "silhouette" (in "cluster" package). To test the method, I used the ...
307 views

### converting continous variable to discrete while maximizing gini

I have a continuous independent variable which is used to explain the dependent binary variable in logistic regression. The model user's requirement is to group this continuous variable to 6 bins. I ...