-1
$\begingroup$

I may be thinking of this incorrectly but what would be the best way to cluster a dissimilarity measure that has direction?

For example, if someone had condition A and condition B each represented by a (n x n) matrix with diagonal=1. A comparative measure could be condition A - condition B which would result in both positive and negative values [-1,1]. This would still be a dissimilarity measure of some sort because values that barely change between the 2 conditions will have smaller values while larger magnitude values would be a greater change. However, the directionality is also important because a positive value would indicate a higher value in condition A relative to B while a negative would be higher in B relative to A.

How could one work with this type of structure? Is it possible to cluster this type of data without ignoring the directionality (i.e. instead of absolute value)?

Below is a simple example using the iris dataset. X_iris is a (150,4) dataframe with 3 classes representing species. y_iris is a vector with species labels.

# Get "conditions"
# ================
# Condition A
# -----------
df_setosa = X_iris.loc[y_iris.compress(lambda id_species:id_species == "setosa").index]
# print(df_setosa.head())
#         sepal_length  sepal_width  petal_length  petal_width
# iris_0           5.1          3.5           1.4          0.2
# iris_1           4.9          3.0           1.4          0.2
# iris_2           4.7          3.2           1.3          0.2
# iris_3           4.6          3.1           1.5          0.2
# iris_4           5.0          3.6           1.4          0.2
# -----------
# Condition B
# -----------
df_versicolor = X_iris.loc[y_iris.compress(lambda id_species:id_species == "versicolor").index]
# print(df_versicolor.head())
#          sepal_length  sepal_width  petal_length  petal_width
# iris_50           7.0          3.2           4.7          1.4
# iris_51           6.4          3.2           4.5          1.5
# iris_52           6.9          3.1           4.9          1.5
# iris_53           5.5          2.3           4.0          1.3
# iris_54           6.5          2.8           4.6          1.5

# Get similarity measure
# ======================
# Condition A
# -----------
df_setosa_similarity = df_setosa.corr()
# print(df_setosa_similarity)
#               sepal_length  sepal_width  petal_length  petal_width
# sepal_length      1.000000     0.742547      0.267176     0.278098
# sepal_width       0.742547     1.000000      0.177700     0.232752
# petal_length      0.267176     0.177700      1.000000     0.331630
# petal_width       0.278098     0.232752      0.331630     1.000000
# -----------
# Condition B
# -----------
df_versicolor_similarity = df_versicolor.corr()
# print(df_versicolor_similarity)
#               sepal_length  sepal_width  petal_length  petal_width
# sepal_length      1.000000     0.525911      0.754049     0.546461
# sepal_width       0.525911     1.000000      0.560522     0.663999
# petal_length      0.754049     0.560522      1.000000     0.786668
# petal_width       0.546461     0.663999      0.786668     1.000000
# ----------
# Comarative
# ----------
df_comparative = df_setosa_similarity - df_versicolor_similarity
# print(df_comparative)
#               sepal_length  sepal_width  petal_length  petal_width
# sepal_length      0.000000     0.216636     -0.486873    -0.268363
# sepal_width       0.216636     0.000000     -0.382822    -0.431247
# petal_length     -0.486873    -0.382822      0.000000    -0.455038
# petal_width      -0.268363    -0.431247     -0.455038     0.000000
$\endgroup$
  • $\begingroup$ A valid similarity matrix must be symmetric and nonnegative, so a difference of similarity matrices wouldn't generally be valid. There are ways to transform it to satisfy these criteria, but whether this would make sense depends on what you're trying to do. Could you say more specifically what conditions A and B mean, and what you're intending to capture by the difference of similarity matrices? Maybe a more concrete example would help. The iris example shows what you're trying to do procedurally, but we need to know the intended meaning. $\endgroup$ – user20160 Dec 20 '18 at 22:14
  • $\begingroup$ The Iris example is just had, because it assumes iris_0 = iris_50 when computing the difference of they matrixes. $\endgroup$ – Anony-Mousse Dec 21 '18 at 0:29
0
$\begingroup$

Hierarchical clustering does not care. It does not use anywhere that distances/similarities are not negative.

Also there is the obvious trick of shifting the values to be non-negative, i.e., new(x)=old(x)-min(old(X))

For mant (but not all) linkages, the result will be the exact same cluster tree, just with the height adjusted accordingly..

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.