I'm developing some software to allow users to perform various kinds of clustering on some data using a pairwise distance matrix (k-medoids is the main method). I would like to allow the user to tune a parameter p
that changes how sensitive the method is to outliers, and I've found a particular log transformation of the distances d
that is doing what I want, at least when p = 20
: y = log_10(p*d + 1)
. It acts to "suppress" large values, such that the algorithm differentiates little between medium and large distances; IE the penalty to include an outlier in a cluster is lessened.
However, no matter the value of p the function never approaches the original distances. I want the clustering to be performed on the regular distances when p
is at its "resting" value (maybe 1, maybe 0, depending on the transformation function used), and I want the clustering to be less and less sensitive to outliers as p
is set further and further from this resting value.
Does anyone have a suggestion for such a transformation? Distances will always be positive, but can be very small and can be zero. I don't care if the function has a horizontal asymptote or not; I'm not particularly fussy beyond the idea that it approaches no transformation at some particular value.
Below is a little mock-up graph showing what I'm looking for. The dotted line is no transformation (p = 1
), and the suppression becomes more extreme as p
is increased to 4.