1
$\begingroup$

The objective of clustering analysis is to group data with similar characteristics in clusters, but in this case, I want to find the most unrelated data to group into clusters. In my particular case, I have 100 weather stations during one year with a 1hr interval, and I want to group the most similar weather stations(I performed this step with K-means using the correlation distance via MATLAB, but I can use R or Python). But now, I want to perform the inverse, i.e., group in the same cluster unrelated (with no correlation) weather stations .

Is this possible? If yes, how? Or, should I use others techniques to execute my idea?

The main idea is to use the groups to prove a concept often used in wind power designated as statistical power smoothing effect (pdf). Basically, there is a statistical power smoothing effect in the wind power data, if you considered data with different features (e.g., different weather conditions). I want to use this grouping to show if I carefully grouping my weather stations (that will be transformed in wind power), then I can minimize the fluctuations. So far I applied k-means algorithm to select the weather stations with the same behavior, and now I want to explore the opposite. Probably, the technique that I need is not clustering, but so far I search and nothing came up.

@Pere: Yes, I also expected what you mentioned. Can you provide some reference to understand how I can compute the correlation inverse distance?

@Dougal: I want diverse groups, as in the normal cluster. To be honest the goal was to split the data in the same number of the "normal" clustering, in my case 9.

@ Pere: Thank you for the example. But I didn't achieve the expected results since, in my case, some clusters are very close to the ones obtained with the "normal" cluster. Probably it is better to try a trial and error test (adding/removing weather stations) to understand the weather stations that I should combine to smooth my signal.

$\endgroup$
  • 1
    $\begingroup$ well, you said it yourself, you want to perform the "inverse", so you can try to take the inverse (algebric) of your correlation matrix $\endgroup$ – agenis Oct 13 '16 at 18:00
  • 1
    $\begingroup$ Because this is such an unusual request, could you please provide more specifics about what this grouping is intended to achieve? It will come down to this question (abstractly): given two candidate groupings, exactly how would you decide which one is a better solution than the other? $\endgroup$ – whuber Oct 13 '16 at 18:53
  • $\begingroup$ You just need to define a "distance" that is inverse of usual distance. It won't fit the definition of distance in algebra or topology but for most clustering methods that shouldn't be a problem. Anyway, I would expect rather strange groups and some instability among groups. $\endgroup$ – Pere Oct 13 '16 at 18:53
  • $\begingroup$ Thank you for your suggestion. To be honest I only know how to perform the inverse of functions! Can you please provide how I can perform in my correlation matrix? or indicate a reference/toolbox? $\endgroup$ – Manuel Oct 13 '16 at 23:36
  • $\begingroup$ Are you really trying to find a single, diverse group? Do you want as many diverse groups as possible? $\endgroup$ – Dougal Oct 13 '16 at 23:46
1
$\begingroup$

To cluster the less similar points instead of the more similar, you just need to change the distance matrix in a way that the more different the points are, the lesser distance are given. A simple way to do this is just to use any decreasing function of the usual distance.

In your case your usual distance is probably 1-correlation (distance 0 for very correlated points and 1 for uncorrelated), so you can use as distance the inverse of 1-correlation or even 1+correlation.

I'll put an example of clustering the most different point using geographical distances - that is, clusters will have the most distant points instead of the closest neighbours.

I'll start with the distances between European cities found in http://www.mapcrow.info/european_travel_distance.html (in my code this matrix is the dataset "distsciutats").

distinv<-(1/distsciutats)
di<-as.dist(distinv)
fiti<-hclust(di,method="complete")
plot(fiti,main="clustering by inverse geographical distance")

clustering by inverse geographical distance

You can see how the most distant cities are clustered together.

Just for comparison, I clustered the same cities using geographical distance, as usual.

 dc<-as.dist(distsciutats)
 fit<-hclust(dc,method="complete")
 plot(fit,main="clustering by geographical distance")

clustering by actual distance

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.