Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hierarchical-clustering]

Hierarchical cluster analysis is a method of cluster analysis which builds, by steps, a hierarchy of clusters, a dendrogram. Most popular is agglomerative hierarchical clustering (HAC) which starts from individual objects and collects them into bigger and bigger clusters.

0
votes
0answers
5 views

Bayesian Hierarchical Clustering: How to calculate probability of Data under $H_1$?

I am working on implementing the Bayesian hierarchical clustering algorithm found here from scratch as a way to practice and learn the algorithm. However, I have hit a snag in calculating the quantity ...
2
votes
1answer
13 views

Formal Definition of Tree-Consistent

I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
0
votes
0answers
15 views

Jaccard Index for Binary Data in R using dist function

I have a presence/absence table of 0s and 1s and I would like to cluster this data. I want to create a pairwise matrix using R's dist function which has a binary ...
0
votes
1answer
27 views

Fixing the maximum distance within a cluster

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
1
vote
1answer
29 views

Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
0
votes
0answers
16 views

Clustering Principal Components

I would like to group principal components based on sample values. That is, for a matrix with columns (PC1, PC2, ... , PCn), and rows with transformed values, I want to group PCs with similar values. ...
0
votes
0answers
8 views

Hierarchical Cluster Analysis

I came across answers to many of questions about Cluster Analysis in this platform. One area I still need clarification has to do with missing cases. I obtained DHS data and extracted sub-regional ...
1
vote
0answers
34 views

Hierarchical clustering dendrogram on a distance matrix

When computing hierarchical clustering over a data matrix, a dissimilarity matrix is first computed in order to build the tree (dendrogram). For example: ...
1
vote
0answers
21 views

What is an appropriate way to re-scale ordinal features, for cluster analysis? AND any thoughts on euclidean-distance for ordinal data?

Background I have data from surveys (on political views from CSES) with answers from respondents in ranking-scales, either 0:10 (0, 1, 2, ..., 10) or 0:3 (0, 1, 2, 3). I want to analyze this data ...
1
vote
0answers
24 views

Cluster analysis on categorical variables

I am trying to group different shark species by the type of gear with which they are caught. So, I have the different shark species, and for each species, I have allocated the different gear types ...
0
votes
0answers
6 views

Dendrogram: Hierachical Clustering on Text data

I would like to use hierarchical clustering for my text data using sklearn.cluster library in Python. However, when I plot the dendrogram to inspect where I should ...
0
votes
0answers
9 views

Shared Frailty Assumptions

Do we need to check the proportional hazards assumption when running a shared frailty model for clustered data?
0
votes
1answer
15 views

Is a difference of means test compromised when the treatment is at a higher level than the unit of measurement?

To paint a scenario, let's say that I'm interested in the impact of a new tool on the speed with which work is performed. I give 100 employees access to this new tool while another 100 use the current ...
0
votes
0answers
13 views

Interpreting the Cophenetic correlation

I have a large dataset, for which I have created a dendogram using the Ward model. Next, I calculated the Cophenetic correlation, which resulted in a value of 0.728....
0
votes
0answers
8 views

Nomenclature of hierarchical clustering constellation plot/graph

What is the name for the process/method by which I select the branch that has that dot, circled in red, from within the hierarchical clustered model? This is a constellation plot, made in JMP: I am ...
0
votes
0answers
30 views

Dendrogram y-axis labeling confusion

I have a large (106x106) correlation matrix in pandas with the following structure: ...
1
vote
0answers
19 views

Practical questions about cluster bootstrap confidence intervals

I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
0
votes
0answers
6 views

Hierarchical clustering based on relative error

How can I use Weka to do hierarchical clustering, but based on the % difference between two elements rather than absolute elements? Let's say I want to draw many circles with specific radii. I have a ...
0
votes
0answers
13 views

Weighting and Clustering in Pooled Data Analysis

I am working with DHS (Demographic and Health Survey) data. I have pooled data from about 25 countries taking 2 most recent waves from each country. My dependent variable is neghaz (negative of height ...
0
votes
1answer
17 views

importance of correlation between data for a PROC CLUSTER

i'm working on a clustering analysis on SAS. I need to improve an actual code : ...
0
votes
1answer
40 views

How to inform the space and time complexity of K-means, SOM and Hierachical clustering

In the paper I am writing, one of the reviewers asked for an "a simple computational complexity analysis or time computational demands of their method" My question is : Can I simply report the ...
1
vote
1answer
15 views

Clustering with constraint on minimum size of cluster

I have dataset of $n$ objects, I want to cluster them according to correlation and I want to divide the dataset into groups of similar objects of sizes not less than 50 - because I use clustering for ...
1
vote
1answer
28 views

What does each cluster represent?

I have a dataset from a questionnaire with over 10000 rows and 30 variables. I am trying to have an insight of the data so I tried to cluster similar items. I first made a dimension reduction ...
0
votes
0answers
24 views

Hierarchical clustering: Should I first normalize / correct for phenotypic data?

I've a set of N=100 samples, each sample having M=10 variables (100x10 matrix). These 10 variables (M_i) are responses to some drugs. In addition I've for each sample a list of phenotype data (...
-1
votes
1answer
29 views

jaccard distance if one samples is a subset

I have a very basic question but I cannot find an answer (especially for a clustering situation). I am trying to do hierarchical clustering of samples using jaccard distances. One sample contains ...
0
votes
0answers
11 views

Multi variable clustering cartographical data

Firstly , my knowledge in statistics is very limited , so excuse me if I'm ask a none well placed question. I have a country ,( example : USA ) , and i have 3 set of data 1) position (lat , lng ) ...
1
vote
1answer
21 views

Can I view the features of my clusters without doing it by hand?

I have performed hierarchical clustering on a data set with 186 participants and 94 variables for each participant. What I want to know is if there is a way to see which features are "driving" my ...
-1
votes
1answer
23 views

How to cluster a (directional) dissimilarity matrix with both positive and negative values?

I may be thinking of this incorrectly but what would be the best way to cluster a dissimilarity measure that has direction? For example, if someone had condition A ...
0
votes
1answer
35 views

R: What is the distance function equivalent for this formula?

Hi I'm using an R package that calculates distance with this formula here, as.dist(1 - cor(df, use = "pa")) However I cannot seem to find an equivalent dist ...
3
votes
1answer
41 views

Why clustering metrics are worse while adding some features?

I am facing something unexpected at first sight and would like to know if you could share some insights. Basically, I have performed a clustering on both qualitative and quantitative data using the ...
0
votes
0answers
40 views

Decision Tree from Agglomerative Clustering

I have agglomerative clustering done. I want to convert it to a decision tree so I can figure out the cluster very quickly. How to do so? A tedious approach (bad, I know): Take the top ...
0
votes
1answer
52 views

Clustering for medium data [closed]

Which clustering method is good in R for a data with ~32,000 subjectsa and 10 variables, hierarchical or k-means?
0
votes
1answer
49 views

How to tell if the “clusters” I see in my pair plots are statistically significant or occurring by chance?

I have a data set with one row per subject. Some variables include laboratory parameters for blood chemistry, hematology, etc. I also have some flag variables: any = 1 if the subject experienced an ...
1
vote
0answers
15 views

How to use a previously calculated set of clusters to start EM Clustering in R?

I have performed Hierarchical Clustering on a data set. I would now like to compare the BIC of the clustering methods. The process involves using the clusters you have determined to act as your ...
2
votes
1answer
195 views

Hierarchical clustering in R - centroid linkage - problem with dendrogram heights

I have got a problem with understanding the hierarchical clustering algorythm, especially centroid linkage method. I have read many articles with description and they seems to be quite easy but I ...
0
votes
1answer
18 views

How to interpret contrasting information from the Variation of Information, Dunn and Rand Index for comparing clusterings

There are related questions but the answers don't seem to explain how to practically judge these measurements for non stats users. I have a dataset which I clustered with K=4 using hierarchical ...
1
vote
2answers
133 views

In cluster analysis should I scale (standardize) my data if variables are in the same units?

I am performing cluster analysis (k-means and hierarchical) based on multiple variables. Each variable is in percentage 0-100% and the sum of all variables is at most 100%. I see that in many of the ...
2
votes
1answer
123 views

Cluster analysis of variables or observations?

I'm very new to cluster analysis. In papers such as Richette et al.1 (which tries to see which concomitant diseases cluster together), authors first cluster the variables and then the observations (i....
1
vote
0answers
20 views

Using pcolor to visualize clusters (from Hierarchical Clustering in multidimensional space) or suggestions? [closed]

I have output from Hierarchical Clustering on my data which is multidimensional and hence difficult to plot and show cluster centers. My data points are regions within a larger image, and I was ...
1
vote
0answers
60 views

How to cluster Categorical/Ordinal Survey Data

I have customer survey response data from Surveymonkey. I want to know what the best method would be for similarity analysis in order to create customer segments. Most of this data is categorical "...
0
votes
0answers
45 views

Accounting for repeat observations in k-means clustering

I am applying k-means clustering (i.e. kmeans function from the stats r package) to a distance matrix generated using ...
1
vote
1answer
86 views

Does it make sense to cluster asymmetric binary data with Ward's method?

I'm working with 104x42 data set where all variables are (asymmetric) binary (0-1). I've read that Ward's linkage method doesn't work theoretically properly with binary data beaucause it requests ...
0
votes
0answers
37 views

Should I use hclustvar instead of hclust to transposed data?

I am performing hierarchical clustering to a dataset with X variables (columns) and Y time points (rows) (kind of multivariate time series), where X << Y. I would like to cluster by the ...
0
votes
0answers
75 views

Computing Gini index and cross-entropy for a dendrogram

I am trying to calculate purity of clusters obtained through hierarchical clustering. I followed this tutorial here: Decision tree learning My question is: is computing these measures the same for ...
-2
votes
1answer
40 views

Aggregating Cluster Analysis Results [closed]

I want to write something about cluster analysis. I have started to read something about it and I faced with a concept which is called ''Aggregating''. What does it mean for cluster analysis? Do I ...
1
vote
1answer
24 views

Hierarchical Cluster Analysis of 100 objects with 114 variables each

I'm intending to make a cluster analysis of 100 objects. I've read a couple of books and determined that a Hierarchical agglomerative procedure with Ward's linkage method should be used in my case. As ...
4
votes
1answer
770 views

Does k-means have any advantages over HDBSCAN expect for runtime?

I have recently learned about HDBSCAN (a fairly new method for clustering, not yet available in scikit-learn) and am really surprised at how good it is. The following picture illustrates that the ...
0
votes
1answer
194 views

silhouette score returns NaN [closed]

I'm computing silhouette_score from sklearn.metrics library in python, for hierarchical clustering. I'm computing this metric for few cuts of the tree (few options of number of clusters, K). For some ...
0
votes
0answers
13 views

Approximation of the number of neighbors a point has within its cluster in ROCK by Sudipto Guha

The paper (ROCK: A Robust Clustering Algorithm for Categorical Attributes) describes an agglomerative clustering algorithm based on the goodness measure for merging clusters $C_i, C_j$ $$g(C_i,C_j)=\...
1
vote
1answer
31 views

Cluster Groups based on overlap

I have the following data: ...