# How to quantify the separability of data?

This is more of a "help me complete my knowledge", rather than a direct question.

Consider the a binary classification of two clouds of point (in general multiple categories, but let's keep it simple for the sake of our conversation).

I am looking for measures of quantifying the clean-ness/well-formed-ness/separability of data points.

Like the first figure below looks more separable that the 2nd one.

Here are a bunch of things I have seen. Could you add more to my list?

• margin:

$$\gamma = \max_{w, b} \min_i y_i (\mathbf{w}.\mathbf{x}_i + b), y_i \in \{\pm 1\}$$

• distance between means: $$d_{centers} = \|\mu_1 - \mu_2\|$$ where $$\begin{cases} \mu_1 = \frac{1}{| \{i: y_i = +1\} |}\sum_{i: y_i = +1} x_i \\ \mu_2 = \frac{1}{| \{i: y_i = -1\} |}\sum_{i: y_i = -1} x_i \end{cases}$$

• Min distance between pairs of points. $$d_{avg} = \frac{1}{|\{i: y_i = 1 \}||\{j: y_j = -1 \}|} \sum_{i \text{ s.t. } y_i = +1, j \text{ s.t. } y_j = -1} \|\mathbf{x}_i - \mathbf{x}_j\|$$

• Average distance between pairs of points. $$d_{\min} = \min_{i \text{ s.t. } y_i = +1, j \text{ s.t. } y_j = -1} \|\mathbf{x}_i - \mathbf{x}_j\|$$