Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hierarchical-clustering]

Hierarchical cluster analysis is a method of cluster analysis which builds, by steps, a hierarchy of clusters, a dendrogram. Most popular is agglomerative hierarchical clustering (HAC) which starts from individual objects and collects them into bigger and bigger clusters.

0
votes
0answers
7 views

What does a high silhouette score for assigning everything to 1 cluster mean?

I'm writing my bachelor's thesis and I'm running into an oddity. When running k-means and hierarchical, the clustering is fairly evenly distributed - there isn't a clear preponderance of data points ...
0
votes
0answers
13 views

How to choose smallest distance when using agglomerative hierarchical clustering?

I'm trying to follow an example which explains how to build a dendrogram using agglomerative hierarchical clustering, single links and the distance function is Manhattan distance. In particular I don'...
1
vote
1answer
14 views

Clustering data with covariance for each point

I am looking to cluster data points that each have a covariance around itself (based on some function of its neighbourhood, but how I got it is not important). I would like to use the covariance to ...
0
votes
2answers
35 views

Machine Learning Algorithm for Count or Visit data

I am trying to figure out a good approach to use some machine learning on doctor appointment data. I want to first do an unsupervised clustering to look for any natural structure within the data (...
0
votes
0answers
5 views

How to get cluster labels based on distance in hierarchical clustering

Anyone knows how to get cluster labels based on distance in hierarchical clustering in R? For example, in the dendrogram, the vertical axis is Height, which I assume to be the distance. But how to get ...
0
votes
1answer
24 views

Generation of synthetic data for Hierarchical clustering

I wanted to test various hierarchical clustering algorithms to check which algorithm performs best. For this, I was considering simulating some ground truth. Is the possible to generate a correlation ...
1
vote
1answer
29 views

Hierarchical clustering for aggregrated features at higher thresholds/levels?

I am trying to use clustering on certain data. The data itself has three natural levels: at the lowest level the elements are fundamental building blocks, at the second level these fundamental ...
0
votes
0answers
21 views

Time series novel

I've exhaustively attempted to find a proper way to analyse a dataset. Despite finding several piece of information of what could be done, I kindly ask for suggestions of could be done, mainly in R. ...
-1
votes
1answer
25 views

Hierarchical clustering in R

I have a dataset of around 25 observations and most of them being categorical. I have three questions for this. 1- Do the covariates I pick for hierarchical clustering matter or should I try and ...
0
votes
0answers
5 views

Truncate a Hierarchical Clustering tree in order to get the cophenetic coefficient

I've gone ahead and clustered a dataset using a Euclidian Hierarchical Clustering algorithm: ...
0
votes
0answers
9 views

Dimensionality reduction before clustering cosine data values causes a change of scale

In my experiment, I am doing hierarchical agglomerative clustering of texts (parameters: cosine, average). My features matrix is very sparse, so I considered PCA as dimensionality reduction technique. ...
0
votes
0answers
6 views

Construct least-squares ultrametric (hierarchical clustering) fits to doubly-standardized “flow” tables and compare to single-linkage-type fits

Figure 1 of the paper, "Hierarchical Migration Regions of France" (IEEE Transactions on Systems, Man and Cybernetics, 4 (1976) 321-324) (https://www.researchgate.net/publication/...
0
votes
0answers
22 views

How to validate clusters after calculating Gower distances and Ward's clustering in R

I am trying to apply Ward's clustering on a mixed types dataset, and wanna explain what I did (maybe helpful to others), and I have some questions regarding this analysis, mainly how to validate my ...
0
votes
0answers
12 views

LCL vs PML + cluster analysis

I'm running a discrete choice experiment, and am exploring two options for the analysis: Conduct a mixed logit, save the individual subject estimates, do a hierarchical cluster analysis of the ...
2
votes
1answer
54 views

What to do when results of hiearchical, k-means elbow, and k-means silhoutte disagree?

I am conducting a cluster analysis involving 60 subjects and 5 continuous variables. After appropriate scaling, I performed hierarchical clustering with Euclidean distance and complete linkage, and ...
0
votes
0answers
14 views

Bayesian Hierarchical Clustering: How to calculate probability of Data under $H_1$?

I am working on implementing the Bayesian hierarchical clustering algorithm found here from scratch as a way to practice and learn the algorithm. However, I have hit a snag in calculating the quantity ...
2
votes
1answer
33 views

Formal Definition of Tree-Consistent

I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
0
votes
0answers
152 views

Jaccard Index for Binary Data in R using dist function

I have a presence/absence table of 0s and 1s and I would like to cluster this data. I want to create a pairwise matrix using R's dist function which has a binary ...
0
votes
1answer
39 views

Fixing the maximum distance within a cluster

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
1
vote
1answer
31 views

Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
0
votes
0answers
18 views

Clustering Principal Components

I would like to group principal components based on sample values. That is, for a matrix with columns (PC1, PC2, ... , PCn), and rows with transformed values, I want to group PCs with similar values. ...
0
votes
1answer
15 views

Hierarchical Cluster Analysis

I came across answers to many of questions about Cluster Analysis in this platform. One area I still need clarification has to do with missing cases. I obtained DHS data and extracted sub-regional ...
1
vote
0answers
116 views

Hierarchical clustering dendrogram on a distance matrix

When computing hierarchical clustering over a data matrix, a dissimilarity matrix is first computed in order to build the tree (dendrogram). For example: ...
2
votes
0answers
35 views

What is an appropriate way to re-scale ordinal features, for cluster analysis? AND any thoughts on euclidean-distance for ordinal data?

Background I have data from surveys (on political views from CSES) with answers from respondents in ranking-scales, either 0:10 (0, 1, 2, ..., 10) or 0:3 (0, 1, 2, 3). I want to analyze this data ...
1
vote
0answers
39 views

Cluster analysis on categorical variables

I am trying to group different shark species by the type of gear with which they are caught. So, I have the different shark species, and for each species, I have allocated the different gear types ...
0
votes
1answer
19 views

Dendrogram: Hierachical Clustering on Text data

I would like to use hierarchical clustering for my text data using sklearn.cluster library in Python. However, when I plot the dendrogram to inspect where I should ...
0
votes
0answers
12 views

Shared Frailty Assumptions

Do we need to check the proportional hazards assumption when running a shared frailty model for clustered data?
0
votes
1answer
20 views

Is a difference of means test compromised when the treatment is at a higher level than the unit of measurement?

To paint a scenario, let's say that I'm interested in the impact of a new tool on the speed with which work is performed. I give 100 employees access to this new tool while another 100 use the current ...
0
votes
0answers
28 views

Interpreting the Cophenetic correlation

I have a large dataset, for which I have created a dendogram using the Ward model. Next, I calculated the Cophenetic correlation, which resulted in a value of 0.728....
0
votes
0answers
9 views

Nomenclature of hierarchical clustering constellation plot/graph

What is the name for the process/method by which I select the branch that has that dot, circled in red, from within the hierarchical clustered model? This is a constellation plot, made in JMP: I am ...
0
votes
0answers
48 views

Dendrogram y-axis labeling confusion

I have a large (106x106) correlation matrix in pandas with the following structure: ...
1
vote
0answers
25 views

Practical questions about cluster bootstrap confidence intervals

I want to estimate the accuracy of a machine learning model. I have an independent test set with a vector of trusted labels and a corresponding vector of model-based predictions. If I assume the test ...
0
votes
0answers
7 views

Hierarchical clustering based on relative error

How can I use Weka to do hierarchical clustering, but based on the % difference between two elements rather than absolute elements? Let's say I want to draw many circles with specific radii. I have a ...
0
votes
0answers
19 views

Weighting and Clustering in Pooled Data Analysis

I am working with DHS (Demographic and Health Survey) data. I have pooled data from about 25 countries taking 2 most recent waves from each country. My dependent variable is neghaz (negative of height ...
0
votes
1answer
24 views

importance of correlation between data for a PROC CLUSTER

i'm working on a clustering analysis on SAS. I need to improve an actual code : ...
0
votes
1answer
61 views

How to inform the space and time complexity of K-means, SOM and Hierachical clustering

In the paper I am writing, one of the reviewers asked for an "a simple computational complexity analysis or time computational demands of their method" My question is : Can I simply report the ...
1
vote
1answer
17 views

Clustering with constraint on minimum size of cluster

I have dataset of $n$ objects, I want to cluster them according to correlation and I want to divide the dataset into groups of similar objects of sizes not less than 50 - because I use clustering for ...
1
vote
1answer
30 views

What does each cluster represent?

I have a dataset from a questionnaire with over 10000 rows and 30 variables. I am trying to have an insight of the data so I tried to cluster similar items. I first made a dimension reduction ...
0
votes
0answers
33 views

Hierarchical clustering: Should I first normalize / correct for phenotypic data?

I've a set of N=100 samples, each sample having M=10 variables (100x10 matrix). These 10 variables (M_i) are responses to some drugs. In addition I've for each sample a list of phenotype data (...
-1
votes
1answer
40 views

jaccard distance if one samples is a subset

I have a very basic question but I cannot find an answer (especially for a clustering situation). I am trying to do hierarchical clustering of samples using jaccard distances. One sample contains ...
0
votes
0answers
11 views

Multi variable clustering cartographical data

Firstly , my knowledge in statistics is very limited , so excuse me if I'm ask a none well placed question. I have a country ,( example : USA ) , and i have 3 set of data 1) position (lat , lng ) ...
1
vote
1answer
24 views

Can I view the features of my clusters without doing it by hand?

I have performed hierarchical clustering on a data set with 186 participants and 94 variables for each participant. What I want to know is if there is a way to see which features are "driving" my ...
-1
votes
1answer
42 views

How to cluster a (directional) dissimilarity matrix with both positive and negative values?

I may be thinking of this incorrectly but what would be the best way to cluster a dissimilarity measure that has direction? For example, if someone had condition A ...
0
votes
1answer
38 views

R: What is the distance function equivalent for this formula?

Hi I'm using an R package that calculates distance with this formula here, as.dist(1 - cor(df, use = "pa")) However I cannot seem to find an equivalent dist ...
3
votes
1answer
44 views

Why clustering metrics are worse while adding some features?

I am facing something unexpected at first sight and would like to know if you could share some insights. Basically, I have performed a clustering on both qualitative and quantitative data using the ...
0
votes
0answers
71 views

Decision Tree from Agglomerative Clustering

I have agglomerative clustering done. I want to convert it to a decision tree so I can figure out the cluster very quickly. How to do so? A tedious approach (bad, I know): Take the top ...
0
votes
1answer
54 views

Clustering for medium data [closed]

Which clustering method is good in R for a data with ~32,000 subjectsa and 10 variables, hierarchical or k-means?
0
votes
1answer
49 views

How to tell if the “clusters” I see in my pair plots are statistically significant or occurring by chance?

I have a data set with one row per subject. Some variables include laboratory parameters for blood chemistry, hematology, etc. I also have some flag variables: any = 1 if the subject experienced an ...
1
vote
0answers
17 views

How to use a previously calculated set of clusters to start EM Clustering in R?

I have performed Hierarchical Clustering on a data set. I would now like to compare the BIC of the clustering methods. The process involves using the clusters you have determined to act as your ...
2
votes
1answer
328 views

Hierarchical clustering in R - centroid linkage - problem with dendrogram heights

I have got a problem with understanding the hierarchical clustering algorythm, especially centroid linkage method. I have read many articles with description and they seems to be quite easy but I ...