0
$\begingroup$

How can I use Weka to do hierarchical clustering, but based on the % difference between two elements rather than absolute elements?

Let's say I want to draw many circles with specific radii. I have a vector of numbers where each number represents the radius of one circle. For each number, I set my compass to that radius using a ruler, draw, and repeat. I notice some radiuses are pretty close. Because it is tedious to set the compass, I decide to just use one approximation for all of them. For example:

  • If I have values $(51.3, 51.7, 51.8)$ they are all pretty similar. I could just set my compass to their mean 51.6, and draw 3 times, which is close enough (the error is only 0.6%, 0.1% and 0.4%) but saves me some work.
  • On the other hand if the values were $(12, 34, 119)$ I definitely want to set the compass each time. If I just use the mean 55, then they would be off by 358%, 61% and 53%. Not really worth it.

So it seems like I want to group similar values, hence I should cluster. But because I'm interested in percent error, I want the distances to be scaled by each element. I couldn't figure out how to do that.

If I just hierarchically cluster the values by Euclidean distance, the high values do fine, but lower values get grouped close together because the Euclidean distance between them is small. I think if instead of calculating distance as $|i-j|$, it was calculated as $2 \frac{|i-j|}{i+j}$, and the hierarchical clustering ran on that, it would give me the result I want. However, I couldn't find a way of implementing this in Weka. I am given options of Euclidean, Manhattan, Chebyshev and Minkowski distances and none of these seems to scale by mean. I couldn't come up with a filter that would do what I want either.

Do I really have to write my own distance function (how?) or am I overlooking something simple?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.