Skip to main content

Questions tagged [hierarchical-clustering]

Hierarchical cluster analysis is a method of cluster analysis which builds, by steps, a hierarchy of clusters, a dendrogram. Most popular is agglomerative hierarchical clustering (HAC) which starts from individual objects and collects them into bigger and bigger clusters.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Combining classification models and regression (Hierarchical? or MoE modeling?)

Assume, there is a dataset of human poses captured from different angles. For each sample there are 2 level of labels - pose type (standing, sitting, etc) and pose magnitute(continuous). Defined tasks,...
PatrickHellman's user avatar
1 vote
0 answers
25 views

Evaluate hierarchical clustering with partial ground truth

I am performing hierarchical clustering, and I need to decide which agglomeration method to use. While I don't have a ground truth, I know that some datapoints should be closer together: for example, ...
Alexlok's user avatar
  • 143
1 vote
0 answers
18 views

Hierarchical Clustering with Large Datasets

I am currently aiming to perform hierarchical clustering for the purpose of customer segmentation. My dataset consists of 217,000 instances with 12-15 features. However, due to memory issues when ...
sammcm998's user avatar
1 vote
1 answer
28 views

Name for a dendrogram showing how variable clustering affects variance explained

This is from Harrell's RMS: I'm sure I have seen a similar dendrogram used to illustrate the effect of variable clustering on $R$ / $R^2$. It was plotted with the x, y axes the other way round to the ...
Mohan's user avatar
  • 919
0 votes
0 answers
14 views

Is there some sort of multilevel KNN/ML model I can use to figure out which users will buy specific products?

I am wondering if there is some sort of multilevel model that I can use to identify likely buyers of specific products or create a lookalike audience. The issue is that I have 1000s of products and ...
ibarbo's user avatar
  • 65
1 vote
1 answer
41 views

Hierarchical clustering of a distance matrix with element weights

I am computing a hierarchical clustering of some geospatial data. I need to add in an element weighting to the approach. My current approach is: I compute temporal cross-correlations between my N ...
JoshD's user avatar
  • 51
1 vote
1 answer
32 views

How to interpret hierarchical clustering results

I have a data frame "Customer_original" with customer data, it has around 50 mixed type variables. I standardize "Customer_original" and put the standardized values into "...
ExchangedVisual111's user avatar
1 vote
0 answers
53 views

How to incorporate p-value information for pairwise distances into clustering?

Consider I have a $n \times n$ distance matrix I want to use for hierarchical clustering, where the distance metric I use ranges from 0-1. I also have a second $n \times n$ matrix that gives me the p-...
Jmmer's user avatar
  • 89
0 votes
0 answers
20 views

Hierarchical cluster analysis with mixed data

I have a dataset consisting of 134 observations and four variables. The dataset consists of answers to a questionnaire. I want to perform a hierarchical cluster analysis on the variables in the ...
Lasse H's user avatar
0 votes
2 answers
101 views

Best cluster approach for dichotomous variables in SPSS

I'm reading contradictory information on the internet and wanted to get some thoughts on a statistical approach to use. I'll use a fictional example for the sake of easy interpretation. I have a set ...
user404377's user avatar
0 votes
0 answers
40 views

Drawing a dendrogram based on a confusion matrix

I have a simple 2D confusion matrix that I normalised for every true class. Now, what would be the best method to visualise this as a dendrogram? I have seen hierarchical agglomerative clustering on ...
Damiaan Reijnaers's user avatar
2 votes
1 answer
48 views

Understanding complete linkage

I was trying to understand the linkage function from scipy and I was confused with the output for this sample code ...
Zoso's user avatar
  • 123
0 votes
0 answers
50 views

How to determine which variables are the most important to define each cluster from a Hierarchical clustering analysis, using R

After searching, reading posts, and trying to solve this issue myself for several days, I think it's time to ask for help. I'm performing a Hierarchical Clustering analysis on a dataset, that includes ...
JLLavin's user avatar
0 votes
0 answers
26 views

How to perform Hierarchical Clustering using centroid method and custom distance metric?

I would like to perform Agglomerative Hierarchical Clustering using the centroid method (defined on this page) and a custom distance metric, probably cosine similarity. In the Scipy docs it says you ...
Rupert Hart's user avatar
0 votes
0 answers
28 views

Bayesian Hierarchical Clustering prior update

I am working through Heller and Ghahramani's "Bayesian Hierarchical Clustering" paper (https://www2.stat.duke.edu/~kheller/bhc.pdf) and things aren't quite working out the way I expect with ...
dataphile8's user avatar
1 vote
0 answers
38 views

Why is HDBSCAN considered a transductive clustering algorithm despite having approximate_predict?

In the documents for HDBSCAN it says the following: Often it is useful to train a model once on a large amount of data, and then query the model repeatedly with small amounts of new data. This is ...
TunaFishLies's user avatar
0 votes
0 answers
19 views

How to model clustered data structure association when all clusters are observed at the same time?

I have data from many recipes I made that describes how different ingredients subgroups grouped in different parent categories (sweet, sour, bitter, umami ) work together to ultimately lead to scores ...
Estimate the estimators's user avatar
0 votes
0 answers
18 views

Peculiar silhouette score is reduced with more clusters

I am working with Python and a set of sensors. I want to use clustering on them. To do that I have used hierarchical cluster with average linkage and euclidean distance as: ...
slow_learner's user avatar
1 vote
0 answers
47 views

Clustering for a meaningful fixed effects model

I have panel data for 5 time periods and about 10,000 geographic units. My dependent variable Share is the share of workers in a specific category, and my descriptors represent various factors that ...
Mikhail's user avatar
  • 97
1 vote
1 answer
36 views

Why is the dendrogram structure different from what I see of my data in a 3D PCoA?

I conducted hierarchical clustering on a dissimilarity matrix computed from a series of species distribution models text and I seem to get different visible structure between the dendrogram and the 3D ...
vanguard605's user avatar
4 votes
2 answers
354 views

Question about Silhouette index calculation using scikit

I am currently working with continuous data measured from different sensors (thermometers and voltmeters). I have a matrix whose columns represent the sensors and the rows are normalized measurements (...
slow_learner's user avatar
0 votes
0 answers
22 views

Discrepancies between AU and BP values in HCA

I conducted a hierarchical cluster analysis in R on a pool of 20 words that have certain common traits. I used hclust(). After, I did multiscale bootstrap resampling using pvclust(). While smaller-...
Ivan 's user avatar
  • 21
0 votes
0 answers
9 views

Alternatives to spatial and temporal aggregation of time series to discover more learnable patterns

Given taxi demand time series of towns in a country. I would like to do demand forecasting. I noticed that when the town's time series is zero inflated the prediction is poor. However, when these ...
Jose_Peeterson's user avatar
0 votes
0 answers
16 views

Clustering of Variables around Latent Variables (CLV) over only qualitative data

I'm reaching out today because I have a concern regarding the clustering approach employed with the CLV method introduced by Vigneau and Qannari in 2003. I've noticed that this method is predominantly ...
lasagna's user avatar
  • 11
0 votes
0 answers
19 views

Design Effect in Cluster Randomised Trials

I want to understand why and how variance inflation factor is taken in account with the formula [1+(m-1)*ICC], where M= Size of clusters and ICC is the intracluster correlation. As I have deduced till ...
Angela Sarkar's user avatar
0 votes
0 answers
13 views

hierarchical clustering linkage

Just want to get verified with hierarchical clustering. Let's assume that you want dissimilarity measure as euclidean distance. Let's say that you have three clusters. Each cluster has many ...
user392987's user avatar
0 votes
1 answer
105 views

Multivariate Longitudinal Multi-level Model: What is the hierarchy structure?

I am interested in running a multivariate multi-level model with longitudinal data, and I'm having a hard time conceptualising the hierarchy levels. My variables are: Multivariate outcome: Quality of ...
Emma L's user avatar
  • 1
4 votes
1 answer
117 views

Combine two estimates of same variable

Suppose I know that $ A = 0.3 \cdot B + 0.7 \cdot C $ And I have these estimates: $$ \hat A = A + \varepsilon $$ $$ \hat B = B + \zeta $$ $$ \hat C = C + \eta $$ For the sake of the argument, $ \...
BlackNinja's user avatar
0 votes
0 answers
11 views

Hierarchial clustering given a list of banking syndicates

I have a list of banking syndicates (groups of banks), and I'm trying to do some hierarchical clustering on it to gather if there is some connection between them (e.g. if lots of syndicates share the ...
apg's user avatar
  • 121
0 votes
1 answer
29 views

In unsupervised learning, is a result of 2 clusters meaningful?

I used both agglomerative clustering and k-means on a dataset and see the results below. Result from agglomerative clustering was demonstrated with silhouette score while kmeans with inertia score. ...
LCheng's user avatar
  • 189
0 votes
0 answers
93 views

Calinski-Harabasz score as a validity index for hierarchical clustering with distance matrix

I have a distance matrix that I want to performed hierarchical clustering with, and find the optimal number of clusters by maximizing the Calinski-Harabasz (CH) score. However, I only know how to ...
user17298847's user avatar
2 votes
1 answer
68 views

Inconsistent cluster indices using hierarchical clustering for time series data

I am currently trying to spatially cluster data that is ordered on a grid. Each point has x and y coordinates as well as a measurement value. These features come from a time series where I analyze ...
Krautsultan's user avatar
0 votes
0 answers
60 views

Is "ward.D" a good agglomeration method in R?

I need to do clustering on a large scale file (~12M rows, 18 features + id index). As a first step, i tried different algorythms in Python with a test sample (40k rows) which gave results (clearly ...
MATHIS MANDINE's user avatar
1 vote
0 answers
41 views

P values of coefficients from ClusterBootstrap GLMs

I am using the following data https://docs.google.com/spreadsheets/d/1ovl8oQnIi_uApVCo93BN_UOUY3F1YoKw/edit?usp=sharing&ouid=117026321594556045652&rtpof=true&sd=true and the ...
user avatar
0 votes
0 answers
359 views

How to automatically choose the optimal number of clusters for a hierarchical clustering dendrogram? (R or Python)

I have a product substitutability matrix, where each cell has a number between 0 and 1 to represent how substitutable (i.e. how similar) the two products are. I then apply hierarchical clustering to ...
Sam Eley's user avatar
1 vote
0 answers
48 views

Finding the nearest point to a given set of points [closed]

I am trying to build an auto-suggestion logic which looks at currently selected items and recommends a list of items to select the next item from. I can formulate the problem thus: Given a set of ...
curious_explorer's user avatar
1 vote
0 answers
84 views

Cluster analysis after factor analysis: What distance measure to use?

I use factor analysis on a set of 15 survey questions (likert scales). Using the predict command (in stata) I make 5 factors. Subsequently, I want to use cluster analysis to see if there are "...
Lotte Yanore's user avatar
1 vote
0 answers
294 views

What are the downsides of using euclidean distance for hierarchical clustering of a correlation matrix?

Apologies if this has been answered elsewhere, but I couldn't find any answers discussing this specific question. I am lacking some notion on clustering using euclidean vs correlation distance, when ...
tomsgoms's user avatar
1 vote
1 answer
219 views

Unsupervised learning: How to identify differences between clusters?

I'm learning about unsupervised learning and I tried to use KMeans, AgglomerativeClustering and DBSCAN on the same datase. The result was ok, they seems to work fine according silhouette_score() ...
Antonio Caipora's user avatar
2 votes
1 answer
304 views

Choosing the best clustering algorithm and evaluating the results

I'm trying to separate my data into clusters using the k-means algorithm and the hierarchical algorithm, choose which algorithm fits my data the best, and evaluate the results. However, all of my ...
Jim's user avatar
  • 61
10 votes
1 answer
400 views

Are there algorithms to cluster Graphs, not just cluster nodes in a graph?

I am wondering if there are algorithms to cluster graphs; what I meant is to cluster many graphs, not cluster the nodes in a graph. For example, we have N graphs, G1, G2, G3, .....GN. Then we can ...
TripleH's user avatar
  • 377
0 votes
0 answers
167 views

Negative gap statistic interpretation for cluster analysis

I am trying to perform a cluster analysis on a dataset. The plot of clusters vs. gap statistic is below. I do not know how to interpret the decrease of the gap statistic and its values which are ...
bobo's user avatar
  • 23
0 votes
1 answer
39 views

Feature Scaling in Hierarchal Clustering

I know that feature scaling is always a requirement for clustering algorithms. Currently I am implementing hierarchal clustering on this dataset, I will use only the annual income and the spending ...
AAA's user avatar
  • 145
3 votes
1 answer
823 views

Using Ward's method on a dissimilarity matrix of Gower distances

I have a question regarding Wards method of hierarchical clustering. I used Gower Distance to create a dissimilarity matrix from an event log. I want to agglomerate it with Ward's method. Lets suppose ...
Spypsyduck's user avatar
0 votes
1 answer
58 views

What graph is best suited for this data? (and how can I produce it in R)

I have this df object (9x19), ...
wantingtoimprove's user avatar
0 votes
0 answers
113 views

Clustering Algorithm with Continuous and Binary Variables

I am creating a clustering analysis with both Continuous and Binary variables and am wondering What type of model is best for these cases How should I scale my data. Should I only scale Continuous ...
BigDawg007's user avatar
1 vote
1 answer
75 views

Distance function that captures both circular and "appear as line" clusters [closed]

based on what I know in k-mean clustering, if i use single linkage distance it can capture clusters of thread shapes but it is not suitable for capturing circular clusters. Also If we use complete ...
user368884's user avatar
2 votes
0 answers
327 views

Multilevel/ Mixed Model / HLM Centering Interactions Level 1 and Level 2 Cross-Level

I am having some trouble with the literature on the correct model specification for my question. Here is the setup: I have a multilevel model with a variety of variables at level 1 and a single level ...
bzh's user avatar
  • 21
1 vote
0 answers
170 views

Can I run a Manova after conducting a cluster analysis?

So my study intends to conduct a cluster analysis using 2 variables (RP and AP). I intend to find four clusters (AC, BC, CC and DC). I would then like to examine differences in education, height, age, ...
Rega's user avatar
  • 11
1 vote
1 answer
523 views

Comparing clustering methods based on internal Cluster Validity Indices

I have used the R package dtwclust to generate clusters for more than a thousand time-series objects.Since I did not have any prior information on the number or validity of clusters, I used a suite of ...
Mansi's user avatar
  • 41

1
2 3 4 5
10