Questions tagged [hierarchical-clustering]

Hierarchical cluster analysis is a method of cluster analysis which builds, by steps, a hierarchy of clusters, a dendrogram. Most popular is agglomerative hierarchical clustering (HAC) which starts from individual objects and collects them into bigger and bigger clusters.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

In cluster analysis, is it better to normalize the data or standardize it?

In cluster analysis, is it better to normalize to $[0, 1]$ (i.e., $\frac{x-\min(x)}{\max(x)-\min(x)}$) the data or standardize via z-score (i.e., $\frac{x-\bar{x}}{s_x}$) it? I know normalization ...
0
votes
0answers
9 views

Using dimension reduction techniques for poverty/wealth indicator

I would like to create an indicator/index of a person's wealth (or socio-economic status, SES). I have about 20 variables that are a combination of education, household assets, access to money, and ...
0
votes
0answers
31 views

How can i choose the best result based on clustering

First of all: Sorry for my English grammer, but i hope you can help me! I have an question after working with the results of the folowing question: Plotting a heatmap given a dendrogram and a ...
0
votes
3answers
28 views

How to identify or give a meaning to the cluster membership in a hierarchical clustering?

I know clustering is a type of unsupervised learning problem, however when Kmean clustering is used one can sort the membership based on the cluster centers. For example consider the cluster ...
0
votes
0answers
46 views

Using NbClust on datasets that produce some negative eigenvalues. When to exclude data, when to force to positive, when to exclude test index?

Background on why I am using clustering: I am analyzing data from a multistep biological experiment, where each step is done in batches of varying sizes. I want to account for any biases that might ...
0
votes
0answers
15 views

R: Attribute Overepresentation in a cluster

I performed hierarchical clustering in a dataset with 4 categorical variables (2 nominal, 2 ordinal) in R using hclust and daisy ...
1
vote
1answer
18 views

Should we most of the time use Ward's method for hierarchical clustering?

By browsing notebooks on the web, I see that most of the time Ward's method is used for hierarchical clustering. What could explain its popularity? Does it mean that in general it performs better than ...
0
votes
1answer
44 views

Hclust in R - Categorical data with multiple categories

I have a dataset with 266 observations with categorical variables of multiple categories. I am using the function hclust in R and the function ...
1
vote
1answer
27 views

Am I right that Calinski-Harabasz index (Pseudo-F) can not be calculated from a distance matrix other than euclidean?

Part: I wonder if one could calculate the Calinski-Harabasz index when only having a distance matrix (and a cluster solution, of course). As you need the within and between sum of squares to come up ...
0
votes
1answer
43 views

distance metric for student course schedules

I'm doing an exploratory clustering analysis of student course schedules at a college. Interpretability by humans is paramount: we're trying to inform future research questions and possibly ...
0
votes
0answers
44 views

Evaluating HCPC clusters using cluster.stats from fpc library

I'm trying to get quality measures for example silhouette from HCPC clusters I can get avg.silwidth from the k-means algorithm, using cluster.stats from fpc library : ...
0
votes
1answer
51 views

Suggestions to cluster more than 300k observations

I am trying to perform an hierarchical clustering on data frame that contains 300k records 7 features (3 binaries and 4 continuous) in order to get insights on what looks like my dataset. I've chosen ...
1
vote
2answers
47 views

Hierarchical Clustering, Why Always Agglomerative?

I'm working on clustering for 6 month right now and there is question that bothers me lately, and that is why in every single resource about hierarchical clustering someone introduced two types of it (...
1
vote
0answers
20 views

How does one apply hierarchical agglomerative clustering when multiple positions are equidistant (non-unique distance matrix)?

I have been following this single/minimum -linkage example to better understand hierarchical agglomerative clustering. I noticed that the entries of d_ij are unique ...
1
vote
0answers
23 views

Hierarchical clustering on principle components for multidimensional scaling

Essentially I have a data set of distant objects in which I've loaded onto factors using a multidimensional scaling technique. From my understanding, the factor loadings only differ between MDS and ...
0
votes
0answers
10 views

Stepwise regression approach to clustering with “both” direction (merges and splits)

Classic approaches for clustering a set of points are either top-down or bottom-up. At high level: Top-down (or Divisive): you start with a single cluster, then split a single point or a bunch of ...
0
votes
1answer
94 views

Difference between agglomerative and divisive clustering in terms of results?

Both methods in Hierarchical clustering have always the same result (number of clusters and instances in the same clusters) and the difference is only the way they use to compute the result? Or the ...
0
votes
0answers
42 views

Is it possible that PCA works better without data scaling? [duplicate]

I am a beginner. I have a dataset of 1700 samples with 4 features and I have to perform Hierarchical Clustering (the agglomerative version) and I need to decide whether or not to scale the data and ...
0
votes
0answers
64 views

OPTICS Reachability Curve: Scikit-learn VS PyClustering implementations

I am comparing the Reachability curves generated by the OPTICS algorithm using the Scikit-learn (new implementation) VS the PyClustering one. The hyper parameters are not the same, but it seems that ...
1
vote
1answer
22 views

How does non-uniqueness of data (aka duplicate data points) affect clustering?

I am trying to self-learn more about different clustering methods. I think I understand the main idea of the algorithms, but perhaps their use-cases can shed light on something that puzzles me - ...
0
votes
0answers
32 views

What does a high silhouette score for assigning everything to 1 cluster mean?

I'm writing my bachelor's thesis and I'm running into an oddity. When running k-means and hierarchical, the clustering is fairly evenly distributed - there isn't a clear preponderance of data points ...
0
votes
0answers
15 views

How to choose smallest distance when using agglomerative hierarchical clustering?

I'm trying to follow an example which explains how to build a dendrogram using agglomerative hierarchical clustering, single links and the distance function is Manhattan distance. In particular I don'...
1
vote
1answer
50 views

Clustering data with covariance for each point

I am looking to cluster data points that each have a covariance around itself (based on some function of its neighbourhood, but how I got it is not important). I would like to use the covariance to ...
0
votes
2answers
47 views

Machine Learning Algorithm for Count or Visit data

I am trying to figure out a good approach to use some machine learning on doctor appointment data. I want to first do an unsupervised clustering to look for any natural structure within the data (...
0
votes
0answers
22 views

How to get cluster labels based on distance in hierarchical clustering

Anyone knows how to get cluster labels based on distance in hierarchical clustering in R? For example, in the dendrogram, the vertical axis is Height, which I assume to be the distance. But how to get ...
0
votes
1answer
37 views

Generation of synthetic data for Hierarchical clustering

I wanted to test various hierarchical clustering algorithms to check which algorithm performs best. For this, I was considering simulating some ground truth. Is the possible to generate a correlation ...
1
vote
1answer
31 views

Hierarchical clustering for aggregrated features at higher thresholds/levels?

I am trying to use clustering on certain data. The data itself has three natural levels: at the lowest level the elements are fundamental building blocks, at the second level these fundamental ...
0
votes
0answers
26 views

Time series novel

I've exhaustively attempted to find a proper way to analyse a dataset. Despite finding several piece of information of what could be done, I kindly ask for suggestions of could be done, mainly in R. ...
-1
votes
1answer
41 views

Hierarchical clustering in R

I have a dataset of around 25 observations and most of them being categorical. I have three questions for this. 1- Do the covariates I pick for hierarchical clustering matter or should I try and ...
0
votes
0answers
34 views

Truncate a Hierarchical Clustering tree in order to get the cophenetic coefficient

I've gone ahead and clustered a dataset using a Euclidian Hierarchical Clustering algorithm: ...
0
votes
0answers
16 views

Dimensionality reduction before clustering cosine data values causes a change of scale

In my experiment, I am doing hierarchical agglomerative clustering of texts (parameters: cosine, average). My features matrix is very sparse, so I considered PCA as dimensionality reduction technique. ...
0
votes
0answers
7 views

Construct least-squares ultrametric (hierarchical clustering) fits to doubly-standardized “flow” tables and compare to single-linkage-type fits

Figure 1 of the paper, "Hierarchical Migration Regions of France" (IEEE Transactions on Systems, Man and Cybernetics, 4 (1976) 321-324) (https://www.researchgate.net/publication/...
1
vote
1answer
77 views

How to validate clusters after calculating Gower distances and Ward's clustering in R

I am trying to apply Ward's clustering on a mixed types dataset, and wanna explain what I did (maybe helpful to others), and I have some questions regarding this analysis, mainly how to validate my ...
0
votes
0answers
12 views

LCL vs PML + cluster analysis

I'm running a discrete choice experiment, and am exploring two options for the analysis: Conduct a mixed logit, save the individual subject estimates, do a hierarchical cluster analysis of the ...
2
votes
1answer
86 views

What to do when results of hiearchical, k-means elbow, and k-means silhoutte disagree?

I am conducting a cluster analysis involving 60 subjects and 5 continuous variables. After appropriate scaling, I performed hierarchical clustering with Euclidean distance and complete linkage, and ...
0
votes
1answer
76 views

Bayesian Hierarchical Clustering: How to calculate probability of Data under $H_1$?

I am working on implementing the Bayesian hierarchical clustering algorithm found here from scratch as a way to practice and learn the algorithm. However, I have hit a snag in calculating the quantity ...
2
votes
1answer
47 views

Formal Definition of Tree-Consistent

I am working my way through this paper on Bayesian Hierarchical Clustering. I keep seeing the phrase tree-consistent. However, it doesn't seem to be defined anywhere in the paper. There is a ...
0
votes
0answers
308 views

Jaccard Index for Binary Data in R using dist function

I have a presence/absence table of 0s and 1s and I would like to cluster this data. I want to create a pairwise matrix using R's dist function which has a binary ...
0
votes
1answer
190 views

Fixing the maximum distance within a cluster

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
1
vote
1answer
36 views

Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
0
votes
0answers
19 views

Clustering Principal Components

I would like to group principal components based on sample values. That is, for a matrix with columns (PC1, PC2, ... , PCn), and rows with transformed values, I want to group PCs with similar values. ...
0
votes
1answer
25 views

Hierarchical Cluster Analysis

I came across answers to many of questions about Cluster Analysis in this platform. One area I still need clarification has to do with missing cases. I obtained DHS data and extracted sub-regional ...
1
vote
0answers
287 views

Hierarchical clustering dendrogram on a distance matrix

When computing hierarchical clustering over a data matrix, a dissimilarity matrix is first computed in order to build the tree (dendrogram). For example: ...
2
votes
0answers
63 views

What is an appropriate way to re-scale ordinal features, for cluster analysis? AND any thoughts on euclidean-distance for ordinal data?

Background I have data from surveys (on political views from CSES) with answers from respondents in ranking-scales, either 0:10 (0, 1, 2, ..., 10) or 0:3 (0, 1, 2, 3). I want to analyze this data ...
1
vote
0answers
96 views

Cluster analysis on categorical variables

I am trying to group different shark species by the type of gear with which they are caught. So, I have the different shark species, and for each species, I have allocated the different gear types ...
0
votes
1answer
57 views

Dendrogram: Hierachical Clustering on Text data

I would like to use hierarchical clustering for my text data using sklearn.cluster library in Python. However, when I plot the dendrogram to inspect where I should ...
0
votes
0answers
20 views

Shared Frailty Assumptions

Do we need to check the proportional hazards assumption when running a shared frailty model for clustered data?
0
votes
1answer
22 views

Is a difference of means test compromised when the treatment is at a higher level than the unit of measurement?

To paint a scenario, let's say that I'm interested in the impact of a new tool on the speed with which work is performed. I give 100 employees access to this new tool while another 100 use the current ...
0
votes
0answers
89 views

Interpreting the Cophenetic correlation

I have a large dataset, for which I have created a dendogram using the Ward model. Next, I calculated the Cophenetic correlation, which resulted in a value of 0.728....
0
votes
0answers
22 views

Nomenclature of hierarchical clustering constellation plot/graph

What is the name for the process/method by which I select the branch that has that dot, circled in red, from within the hierarchical clustered model? This is a constellation plot, made in JMP: I am ...