How to cluster and visualise vectors of which the components are class indices?

Question

Let's say I have a dataset $\boldsymbol{\mathcal{X}}$ of $N$ samples wherein each sample $\boldsymbol{x}^{(i)}\in \mathcal{X}$, $i \in {1 \ldots N}$, is described by a set of $D$ features, such that $\boldsymbol{x}^{(i)} \in \mathbb{R}^D$, and each component $d$ of $\boldsymbol{x}^{(i)}$, $d \in {1 \ldots D}$, corresponds to a class index corresponding to that feature, i.e., $x^{(i)}_d \in \mathbb{Z}^+$ and could range depending on the number of different classes corresponding to feature $d$.

How can I then visualise elements in this 'space'? Something like PCA does not make sense, as the vectors are just 'holders' of class indices, they are not some representation that really live in a $D$-dimensional space. Right? And as a second question, how to then cluster vectors based on their similarity?

One thing I came up with: for each sample, for each feature, give a point to all other samples for which the class index overlaps (i.e., where $x^{(i)}_d = x^{(j)}_d, i \neq j$). Then you have some sort of similarity measure which you can then possibly exploit to form clusters.

Another idea is to convert all sample vectors to a very big vector that is essentially a concatentation of $D$ one-hot vectors. Though I am not sure whether that is a good option either, because a procedure like PCA or a clustering algorithm does not really know where one 'portion' (corresponding to one feature) in that big vector 'ends', as it threats all dimensions equally, not as belonging to one class.

A third idea is to do something in the realms of 'hierarchical splitting', but I do not really have worked that out yet.

I feel like this is a very elemental question, yet, I could not really find the answer (yet).

Answer 1

As mentioned by @Christian Henning in the comments on the original post, it was indeed Multiple Correspondence Analysis (MCA) I was looking for, making the answer to my post a very simple and short one.

Answer 2

Another option here is to define a distance on the vectors. If they are ordinal categories this would be: $$\sqrt{\sum{(x^i_d - x^j_d)^2}}$$ and if they are not it would be $$\sqrt{\sum{I_{x^i_d = x^j_d}}}$$

Given this metric, you can use a clustering algorithm, make a nearest neighbour model, and visualize with e.g. t-SNE. If you convert the metric to a kernel you can also build SVMs.