1
$\begingroup$

I have a list of electrical feeders. I want to cluster them by their topological characteristics: voltage level, total length, % of underground cable, state of the neutral.

I first made a manual clustering by dividing the feeders in two groups, which are defined for their voltage level (15 kV and 20 kV). Then I applied a normalization process (with max-min normalization formula, see: How to normalize data to 0-1 range?).

So there's the first question: Should I make a normalization process for each group defined by voltage level (15 kV and 20 kV groups)? (I did so, because then I used clustering techniques for each group.)

Then I tested k means, k medoids, and agglomerative hierarchical clustering, to find which algorithm performs best for my dataset. I calculated the average and global silhouette coefficients to estimate the best technique and to choose and optimal number of clusters.

I wrote code to calculate these coefficients for each value of k, from 2 to 25. Due to the heuristic nature of k-means and k-medoids, I ran the calculation for 50 times to obtain a medium value for each value of k. For example, here are the results according to the average silhouette coefficient for the 15 kV feeders group:

enter image description here

I chose a minimum value of k (signed by black dashed line), then I defined k = 12 as an optimum value to not have a huge number of clusters. The algorithm that best performs at k=12 is k-medoids algorithm.

It's worth evaluating if the coefficients converge to a specific number of clusters, so that a robust solution will be found. It's also important to make a visual inspection of the clusters, to find anomalies that are physically not correct.

I wanted to ask if this process is correct.

$\endgroup$
  • $\begingroup$ So is the issue that voltage level is a binary variable, & you want to force it to be included in the clustering? $\endgroup$ – gung Dec 18 '16 at 15:27
  • $\begingroup$ No, it isn't. I don't want to include voltage level in clustering process, but I'd know if a separate min-max normalization has to be made for the feeders in 15 kV group and 20 kV group. I mean, I think that the normalization should be done considering the specific dataset that has to be clustered. $\endgroup$ – Railectric Dec 18 '16 at 16:05
  • $\begingroup$ However, there's another issue. Between the topological characteristics there's the state of the neutral, which hasn't a numeric meaning but it is non-numeric. So I changed it in a number, and I ordered the values according to the technological improvement about the grounding of the neutral (in electrical distribution network), then normalization is made also for this parameter. $\endgroup$ – Railectric Dec 18 '16 at 16:09
  • $\begingroup$ This is a little hard to follow. Can you post an example dataset? Can you say more about your goals for the clustering? Why do you want to cluster these data? What do you hope to do w/ the clustering? What do you want to learn? $\endgroup$ – gung Dec 18 '16 at 16:11
  • $\begingroup$ I've a list of electrical feeders and i want to cluster them according to the following parameters: voltage level, total length of the feeder % of underground cable of the feeder, state of neutral. I want to cluster them to define representive feeders, because the total amount of the feeders is about 21000. With the clustering I hope to obtain well separated classes of feeders, and then I'll study the performance of feeders for each cluster, according to power quality issues. $\endgroup$ – Railectric Dec 18 '16 at 16:17
2
$\begingroup$

So from the plot, clustering did not really work.

The choice of k is completely arbitrary. k=4 is probably your best choice. k=12 is just an average result.

Most likely, the way you preprocessed the data was bad. Normalizing evrything to $[0;1]$ sometimes works, but more often than not you need to spend a lot more time on the process of making the features comparable prior to clustering.

$\endgroup$
  • $\begingroup$ Thank you for the answer. What do you mean for making features comparable? Are there other techniques of preprocessing? $\endgroup$ – Railectric Dec 18 '16 at 16:28
  • $\begingroup$ Yes, plenty. Try approaching it from statistics: what would be a good preprocessing. $\endgroup$ – Anony-Mousse Dec 18 '16 at 22:58
  • $\begingroup$ Then, I have a non-numeric variable so I changed it in a number (0,1,2,3,4), and I ordered the values according to the technological improvement that this variable means. Is it correct? $\endgroup$ – Railectric Dec 18 '16 at 23:52
  • $\begingroup$ There is no such thing as correct or wrong. These are decisions, and they can improve results, or not. Best if you try to find a statistical argument why choice A is supposed to be better than B rather on your data, rather than guessing. There is an infinite number of options to try. $\endgroup$ – Anony-Mousse Dec 19 '16 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.