# Hierarchical clustering of a distance matrix with element weights

I am computing a hierarchical clustering of some geospatial data. I need to add in an element weighting to the approach.

My current approach is: I compute temporal cross-correlations between my N locations and then converting the resulting NxN correlation matrix, C, into a distance matrix D:

D=sqrt(1-C) [following this discussion]

Then I cluster D into K clusters using scipy, minimising the maximum distance between points in a cluster:

D_condensed= np.concatenate([row[i+1:] for i, row in enumerate(D)])


However, as my N data points represent a MxL latitude-longitude grid, the density of points is much higher near the pole than at the equator. The result is the algorithm favours small polar clusters and larger equatorial clusters. Therefore I would like to add a weighting equal to the area represented by each gridpoint, so that the clustering algorithm preferentially favours equatorial gridpoints.

This would be represented by a length N array of weights W.

An example of the problem when using the unweighted case:

• Would it be correct to understand your problem as resulting from an effective oversampling of polar regions due to the use of lat-lon to generate the grid? Regardless, it would help to explain what you hope this clustering will accomplish, because that would suggest possible alternatives or corrections you can apply.
– whuber
Commented Apr 19 at 21:54

• The GIS site has (literally) thousands of threads about this topic, going far beyond the SO thread you reference, so I would suggest it would serve as a better resource for this subject.//Several re-readings of the original post suggest to me the problem is not directly related to the projection, but rather arises from how the earth's surface was partitioned into grid cells. A reprojection would help only if the raw data--before computing the correlation matrix $C$--could be adequately resampled, which brings about many more problems to solve... .