# Clustering unrelated (with no correlation) data

The objective of clustering analysis is to group data with similar characteristics in clusters, but in this case, I want to find the most unrelated data to group into clusters. In my particular case, I have 100 weather stations during one year with a 1hr interval, and I want to group the most similar weather stations(I performed this step with K-means using the correlation distance via MATLAB, but I can use R or Python). But now, I want to perform the inverse, i.e., group in the same cluster unrelated (with no correlation) weather stations .

Is this possible? If yes, how? Or, should I use others techniques to execute my idea?

The main idea is to use the groups to prove a concept often used in wind power designated as statistical power smoothing effect (pdf). Basically, there is a statistical power smoothing effect in the wind power data, if you considered data with different features (e.g., different weather conditions). I want to use this grouping to show if I carefully grouping my weather stations (that will be transformed in wind power), then I can minimize the fluctuations. So far I applied k-means algorithm to select the weather stations with the same behavior, and now I want to explore the opposite. Probably, the technique that I need is not clustering, but so far I search and nothing came up.

@Pere: Yes, I also expected what you mentioned. Can you provide some reference to understand how I can compute the correlation inverse distance?

@Dougal: I want diverse groups, as in the normal cluster. To be honest the goal was to split the data in the same number of the "normal" clustering, in my case 9.

@ Pere: Thank you for the example. But I didn't achieve the expected results since, in my case, some clusters are very close to the ones obtained with the "normal" cluster. Probably it is better to try a trial and error test (adding/removing weather stations) to understand the weather stations that I should combine to smooth my signal.

• well, you said it yourself, you want to perform the "inverse", so you can try to take the inverse (algebric) of your correlation matrix – agenis Oct 13 '16 at 18:00
• Because this is such an unusual request, could you please provide more specifics about what this grouping is intended to achieve? It will come down to this question (abstractly): given two candidate groupings, exactly how would you decide which one is a better solution than the other? – whuber Oct 13 '16 at 18:53
• You just need to define a "distance" that is inverse of usual distance. It won't fit the definition of distance in algebra or topology but for most clustering methods that shouldn't be a problem. Anyway, I would expect rather strange groups and some instability among groups. – Pere Oct 13 '16 at 18:53
• Thank you for your suggestion. To be honest I only know how to perform the inverse of functions! Can you please provide how I can perform in my correlation matrix? or indicate a reference/toolbox? – Manuel Oct 13 '16 at 23:36
• Are you really trying to find a single, diverse group? Do you want as many diverse groups as possible? – Dougal Oct 13 '16 at 23:46

To cluster the less similar points instead of the more similar, you just need to change the distance matrix in a way that the more different the points are, the lesser distance are given. A simple way to do this is just to use any decreasing function of the usual distance.

In your case your usual distance is probably 1-correlation (distance 0 for very correlated points and 1 for uncorrelated), so you can use as distance the inverse of 1-correlation or even 1+correlation.

I'll put an example of clustering the most different point using geographical distances - that is, clusters will have the most distant points instead of the closest neighbours.

I'll start with the distances between European cities found in http://www.mapcrow.info/european_travel_distance.html (in my code this matrix is the dataset "distsciutats").

distinv<-(1/distsciutats)
di<-as.dist(distinv)
fiti<-hclust(di,method="complete")
plot(fiti,main="clustering by inverse geographical distance")


You can see how the most distant cities are clustered together.

Just for comparison, I clustered the same cities using geographical distance, as usual.

 dc<-as.dist(distsciutats)
fit<-hclust(dc,method="complete")
plot(fit,main="clustering by geographical distance")