0
$\begingroup$

Let's say I have a dataset $\boldsymbol{\mathcal{X}}$ of $N$ samples wherein each sample $\boldsymbol{x}^{(i)}\in \mathcal{X}$, $i \in {1 \ldots N}$, is described by a set of $D$ features, such that $\boldsymbol{x}^{(i)} \in \mathbb{R}^D$, and each component $d$ of $\boldsymbol{x}^{(i)}$, $d \in {1 \ldots D}$, corresponds to a class index corresponding to that feature, i.e., $x^{(i)}_d \in \mathbb{Z}^+$ and could range depending on the number of different classes corresponding to feature $d$.

How can I then visualise elements in this 'space'? Something like PCA does not make sense, as the vectors are just 'holders' of class indices, they are not some representation that really live in a $D$-dimensional space. Right? And as a second question, how to then cluster vectors based on their similarity?

One thing I came up with: for each sample, for each feature, give a point to all other samples for which the class index overlaps (i.e., where $x^{(i)}_d = x^{(j)}_d, i \neq j$). Then you have some sort of similarity measure which you can then possibly exploit to form clusters.

Another idea is to convert all sample vectors to a very big vector that is essentially a concatentation of $D$ one-hot vectors. Though I am not sure whether that is a good option either, because a procedure like PCA or a clustering algorithm does not really know where one 'portion' (corresponding to one feature) in that big vector 'ends', as it threats all dimensions equally, not as belonging to one class.

A third idea is to do something in the realms of 'hierarchical splitting', but I do not really have worked that out yet.

I feel like this is a very elemental question, yet, I could not really find the answer (yet).

$\endgroup$
3
  • 2
    $\begingroup$ If I understand this correctly you are basically asking how to visualise multivariate categorical data, aren't you? Have a look at multivariate correspondence analysis. There are also 2-d techniques such as Mosaic plots that could be put together in a matrix plot for more than 2-d data. $\endgroup$ Feb 1, 2023 at 14:16
  • $\begingroup$ Yes! Exactly! Sorry, that is a much better term to describe my problem. Okay, thank you, I will! $\endgroup$ Feb 1, 2023 at 14:17
  • $\begingroup$ Are all the $N$s the same (the vectors have the same length)? Is there a finite set of category labels, eg, each $x$ is a group of patients who each have a disease yes or no? In general, can you say more about your data? Can you provide a small example dataset for people to work with? $\endgroup$ Feb 2, 2023 at 19:15

2 Answers 2

1
$\begingroup$

As mentioned by @Christian Henning in the comments on the original post, it was indeed Multiple Correspondence Analysis (MCA) I was looking for, making the answer to my post a very simple and short one.

$\endgroup$
0
$\begingroup$

Another option here is to define a distance on the vectors. If they are ordinal categories this would be: $$\sqrt{\sum{(x^i_d - x^j_d)^2}}$$ and if they are not it would be $$\sqrt{\sum{I_{x^i_d = x^j_d}}}$$

Given this metric, you can use a clustering algorithm, make a nearest neighbour model, and visualize with e.g. t-SNE. If you convert the metric to a kernel you can also build SVMs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.