Questions tagged [hierarchical-clustering]

Hierarchical cluster analysis is a method of cluster analysis which builds, by steps, a hierarchy of clusters, a dendrogram. Most popular is agglomerative hierarchical clustering (HAC) which starts from individual objects and collects them into bigger and bigger clusters.

Filter by
Sorted by
Tagged with
0 votes
1 answer
25 views

Clustering similar words in different sets

Suppose I have $N$ sets as follow: $$S_1 = \{a_1,a_2,\cdots,a_n\}$$ $$S_2 = \{b_1,b_2,\cdots,b_n\}$$ $$S_3 = \{c_1,c_2,\cdots,c_n\}$$ $$\vdots$$ elements of each set (i.e., $a_n$, $b_n$, $c_n$ and etc)...
1 vote
1 answer
3k views

Different hierarchical clustering results

I'm running a hierarchical clustering on a sample of data using the steps below: ...
1 vote
1 answer
34 views

Comparing clustering methods based on internal Cluster Validity Indices

I have used the R package dtwclust to generate clusters for more than a thousand time-series objects.Since I did not have any prior information on the number or validity of clusters, I used a suite of ...
1 vote
1 answer
242 views

Unsupervised soft clustering methods

I have a D-dimensional dataset composed of exactly two clusters (this is known) for which I have no labels; the clusters can potentially be wildly imbalanced. I'm after a soft (or fuzzy) clustering ...
3 votes
1 answer
380 views

How to specify K cluster in Hierarchical clustering with noisy data?

I'm new in Mining and Clustering and I wonder how to cut off the hierarchical clustering Dendrogram to obtain a specific number of clusters. The problem is here that the data is noisy and the ...
1 vote
1 answer
165 views

How to look into Agglomerative linkage matrix to find optimal number of clusters using python

I am generating the linkage matrix on 73k data-points. The objective is to find the cut-off point by visualising the distance ...
1 vote
0 answers
27 views

Do I correctly apply hierarchical clustering and K-means on the resource-selection-function values?

I'm trying to find the best way to classify bivariate point patterns in spatstat according to the relationship between two point species: Point pattern ...
0 votes
0 answers
14 views

How can I code, process, and quantify the features of a hierarchical network in a scheme that admits siblingship but also auntship links?

I am new to network analysis and I am currently facing a challenge with coding, processing, and quantifying networks in a hierarchical scheme. In this scheme, nodes pertain to differing hierarchical ...
1 vote
0 answers
10 views

Should mixed-data (incl. weights) be pre-normalised/scaled/standardised before computation of a distance matrix for hierarchical clustering?

should i scale the mixed data type before i perform hierarchical clustering? A question very similar to mine was asked above however it wasn't quite answered... For background: I am using a large ...
1 vote
1 answer
35 views

Scale the data for cluster analysis

I have a time-series data. I would like to use (hierarchical) cluster analysis for them. I read that I need to scale my data. My question is, can I use empirical function (transfer the data into (0,1))...
2 votes
1 answer
169 views

Distribution of conditional posterior for Gibbs sampling

The following is a description of how the authors (Yongning Wang & Ruey S. Tsay) of this (2019) paper Clustering Multiple Time Series with Structural Breaks want to perform Gibbs sampling to ...
0 votes
1 answer
19 views

how to calculate the missclassification rate/ Accuracy rate for clustering?

I am working with cluster models. I tried to find an accuracy rate/misclassification rate manually. I found an example that confuses me. The example can be accessed from here. I think the author find ...
1 vote
0 answers
8 views

Determining the significance of left or right origin in a hierarchical lineage tree

I have a phylogenetic tree ( hierarchical lineage tree ). At the end of the branch, each sample is assigned, and each sample originated from either left or right. What I want to do is, at which level ...
0 votes
0 answers
15 views

Measuring Similarity of Multidimensional Time Series

Suppose I have a non-linear time series comprised of 100 timesteps, within each I have 4 features for each of 50 observations. The features are not independent of eachother and the relationships ...
1 vote
1 answer
1k views

How to validate clusters after calculating Gower distances and Ward's clustering in R

I am trying to apply Ward's clustering on a mixed types dataset, and wanna explain what I did (maybe helpful to others), and I have some questions regarding this analysis, mainly how to validate my ...
0 votes
0 answers
12 views

Approximately Unbiased P-value vs Bootstrap Probability: which one should i choose?

Some references first: How is approximately unbiased bootstrap better than a regular bootstrap with regards to hierarchical clustering? Suzuki et al. 2004 https://www.researchgate.net/publication/...
0 votes
0 answers
5 views

Options for investigating hierarchical covariance structures

In many distinct contexts in biology, one covariance structure will emerge in a dataset of some kind if another covariance structure is present. As an example, we may consider proteins produced by ...
0 votes
0 answers
15 views

Is there a hypothesis test for members of hierarchical clustering clusters?

Suppose my data has $N$ observations, and each observation may be classified into one of three factor levels $\{A, B, C\}$. Using a vector of attributes for each observation, $\{x_1, x_2, \ldots, x_n\...
2 votes
1 answer
48 views

Why not link features instead of selecting them - Clustering

Currently, I am working on customer segmentation using their purchase data. I plan to use clustering techniques. So, my data has below info for each customer (9 features and 1 id field) Now I am ...
0 votes
1 answer
313 views

Comparing different linkage methods in hierarchical clustering

Im trying different linkage methods for my hierarchical clustering problem. Now I would like to evaluate which one works better. Is this as easy as just just comparing the two Dunn's index values? Or ...
0 votes
0 answers
20 views

Hierarchical clustering output 99% belongs to a cluster group

I've just done a clustering analysis using hierarchical clustering analysis in Python but the result is not what I expect. Most of them (483/485) belongs to group 1 and the rest to group 0. Is there ...
3 votes
1 answer
48 views

Distance between two clusters after their joining in centroid linkage

For a distance between two clusters A and B of objects given by $d_{AB}=\left \|{m_{A}-m_{B}} \right \|^{2}$ , where $m_{A}$ is the mean of the objects in cluster $A$, show that the formula ...
1 vote
1 answer
252 views

fixed effects model with upper level predictors and cross-sectional data

I am using cross-sectional data with the following OLS model: $$ Y_{(i,j)} = \beta_{(0)} + \beta X_{(i)} + \beta X_{(i,j)} + \beta fixed\; effects_{(j-1)} + \varepsilon_{i,j} $$ where $i$ stands for ...
0 votes
0 answers
25 views

Does the distance matrix have to be symmetric for a cluster analysis using Ward's method?

Does a distance matrix used in a cluster analysis need to be symmetrical? My past experience with cluster analysis usually involved a lower triangular distance matrix but a recent method I came across ...
2 votes
1 answer
247 views

Agglomerative clustering with must-link constraints

How can implement Agglomerative clustering be implemented to support must-link constraints? I have seen the scikit-learn clustering toolkit interesting, but it seems that scikit-learn only supports ...
0 votes
2 answers
3k views

What is the time and space complexity of single linkage hierarchical clustering?

I have read everywhere that the time complexity of hierarchical agglomerative clustering is $\mathcal{O}(n^3)$ and it can be brought down to $\mathcal{O}(n^2 \log n)$. How do we arrive at such ...
1 vote
1 answer
5k views

Difference between Ward hierarchical clustering and K-Means for classification

I have a dataset where of socio-demographic features of a population (expressed as percentages over the total population of the municipality: e.g. 12% of freelancers, 5% of unemployed etc.), each ...
1 vote
0 answers
142 views

How to compare consistency between clustering results and list of values with different levels in R?

I found similar subjects on the website but I may have missed the relation with my own question. I'v seen questions about comparison of clustering results, but here it's more about comparing two lists ...
1 vote
1 answer
67 views

Intuitive explanation of Ward's method

I got this explanation of the Ward's method of hierarchical clustering from Malhotra et. al (2017), and I don't really get what it means: Ward’s procedure is a variance method which attempts to ...
0 votes
1 answer
26 views

Why does single linkage create loose clusters when it uses smallest distance between two points?

The definition of single linkage says: In single linkage method, the distance between two clusters is defined as the minimum distance between two data points in each cluster. However, different ...
0 votes
1 answer
796 views

How to inform the space and time complexity of K-means, SOM and Hierachical clustering

In the paper I am writing, one of the reviewers asked for an "a simple computational complexity analysis or time computational demands of their method" My question is : Can I simply report the ...
0 votes
1 answer
1k views

Splits in Decision Trees vs Dendrograms

gradient boosting is a supervised learning algorithm that splits/grows decision trees to improve predictions iteratively. hierarchical clustering is an unsupervised learning algorithm that splits/...
0 votes
0 answers
32 views

Are height numbers meaningful when interpreting a cluster dendrogram? (Bray Curtis & ward.D2)

When interpreting a dendrogram produced with a Bray Curtis dissimilarity matrix, do the height numbers have more meany than just relative distance between clusters? Based on the dendrogram below and ...
0 votes
1 answer
2k views

Fixing the maximum distance within a cluster

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
0 votes
0 answers
63 views

How to Determine Cluster Count with Hierarchal Clustering

I have analyzed the following Data set in R using hierarchical analysis, and plotted the results. Ive been asked to find and highlight the number of clusters present, but how exactly do you determine ...
0 votes
0 answers
43 views

Strange cluster assignment at edge of cluster

I'm experimenting with HDBSCAN and am encountering some results that I don't fully understand. Mainly about a datapoint that gets assigned to another cluster than I would expect it to be. Hopefully ...
0 votes
0 answers
14 views

Compare hierarchical clustering results with different imporance scaling

For an assignment at university, I am using hierarchical clustering of a dataset of 5 different parameters. The parameters all have different magnitudes, so I standardized the data using a z-score ...
0 votes
0 answers
16 views

Want to understand dendogram's basic expample from scipy website

I like to understand, how the given array leads to corresponding points in x and y coordinates? ...
1 vote
0 answers
229 views

Python Implemenatation of SPSS's Two-Step Clustering

I want to perform a clustering on data with ~40 binary features. I was recommended the two-step approach by Chiu et al.. They basically use a BIRCH variant to determine pre-clusters and then perform ...
1 vote
0 answers
25 views

Similarity between datasets A and B where B is a subset of A

I have two datasets A and B, and for each entry in both datasets I have a mixture of ordered and unordered categorical variables such as gender, age (integral value) and date. It is believed that ...
0 votes
0 answers
69 views

Breaking Tie with Ward's Method Hierarchical Clustering

Viewing the set of single-feature observations below, I think its obvious that the appropriate Euclidean distance-based (e.g., Ward's method) number of clusters is 3. ...
0 votes
0 answers
12 views

Testing individual effects on aggregate dependent variable

Please help! How do you test the effect of individual observations on an aggregate dependent variable? Can HLM or some other multilevel mixed effects model help me test this? I know these models can ...
1 vote
0 answers
28 views

Optimal number of clusters in gene expression data

I'm clustering genes on gene expression data. Here's a hierarchically clustered heatmap using ward linkage and Euclidean distance It clearly shows there are 5 or 6 clusters. Now when I evaluate their ...
2 votes
1 answer
575 views

How does the coloring of dendrograms in SciPy work?

So I am clustering my data using linkage extensions. When I plot the diagrams of the dendrograms scipy chooses to color branches in different colors according to a "color threshold". As I ...
1 vote
0 answers
22 views

Does it make sense to use variables' product as a new variable in a clustering procedure?

I'm trying to separate different groups based on values from width and length using k-means and hierarchical clustering. My question relates to the possibility of using the area — measured as width * ...
2 votes
0 answers
79 views

Are the distances on a hierarchical clustering dendrogram in the same units as the input distance matrix?

I use Aitchison distance as the input to a hierarchical clustering dendrogram. I started labeling and interpreting the dendrogram but wasn't sure about a few aspects: Are the vertical distances on ...
3 votes
1 answer
115 views

Why clustering metrics are worse while adding some features?

I am facing something unexpected at first sight and would like to know if you could share some insights. Basically, I have performed a clustering on both qualitative and quantitative data using the ...
0 votes
0 answers
60 views

Interpretation of the height of a dendrogram and intra/inter deviance corrispondence

I am practicing hierarchical clustering and I am having doubts about the interpretation of dendrogram height. Let's take the code in example with points (1,1), (2,2), (5,5), (9,9), (10,10), and (13,13)...
0 votes
1 answer
41 views

Hamming distance, ignoring matching 0

I have a dataset with about 800 observations, each with about 2000 boolean variables. I would like to cluster the observations. I am using scipy in Python. For a (dis)similarity measure I've chosen ...
51 votes
2 answers
89k views

Choosing the right linkage method for hierarchical clustering

I am performing hierarchical clustering on data I've gathered and processed from the reddit data dump on Google BigQuery. My process is the following: Get the latest 1000 posts in /r/politics ...

1
2 3 4 5
9