I am trying to get something like a confusion matrix for different classes, but without training a model.

The idea is to use some kind of distance between classes.

The data set is like this, just for better understanding


A    B    C ...  YZ
500  3    2 ...  43
500  15   1 ...  2
300  23   9 ...  68

Class 2

A    B    C ...  YZ
100  56   2 ...  58
100  35   6 ...  34
300  23   9 ...  1

Class N

The classes have different row number.

I have only one idea so far - Calculate means for each column separately (one resulting row) and measure distances between these rows

What is the right way to calculate the similarity of classes? Maybe a python library for that?

  • $\begingroup$ As you are anew contributor here is a hint: if an answer actually answers the question, you should mark it as "accepted" so that it is removed from the list of unanswered questions. $\endgroup$ – cdalitz Nov 27 '20 at 14:18

Your suggestion of summing up the (squared) distances between the class means actually is an established method and is the trace of the between scatter matrix:

$$S_B = \sum_{i=1}^{C} n_i (\vec{\mu}_i - \vec{\mu})\cdot (\vec{\mu}_i - \vec{\mu})^T$$

It is however more useful to set this value in relation to the spread inside the classes, or the trace of the within scatter matrix:

$$S_W = \sum\limits_{i=1}^{C} \left(\sum\limits_{\vec{x}\in\omega_i}(\vec{x} - \vec{\mu}_i)\cdot (\vec{x} - \vec{\mu}_i)^T\right)$$

Measures for class separability are thus

$$J_1 = \frac{trace(S_B)}{trace(S_W)} \quad\mbox{ or }\quad J_2 = trace\Big(S_W^{-1} \cdot S_B\Big)$$

where trace stands for the sum of the diagonal elements, which represent the variances.

The normalization with $trace(S_W)$ makes the measure scale invariant and measures how large the distance between the classes is compared to the distance within the classes.

  • $\begingroup$ Thank you very much for your answer! I really appreciate your attention to my question. Could you please explain me the difference between the approaches, why one is better than another? $\endgroup$ – Statsnewbie Nov 27 '20 at 14:11
  • $\begingroup$ Which approaches do you mean? $J_1$ and $J_2$? These do not make a great difference, I think. Or do you mean the normalization with $trace(S_W)$ compared to your approach that only measures $trace(S_B)$? I have added an explanation for this normalization. $\endgroup$ – cdalitz Nov 27 '20 at 14:15
  • $\begingroup$ Thank you for help, your answer makes me happy 😊 $\endgroup$ – Statsnewbie Nov 27 '20 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.