Clustering on Data with Linear but Poor separability

Question

I'm interested in building an unsupervised clustering model to look at the physical activity data of a person. Below is an image of the model in the 2D space (built through t-SNE), where each data point has been labelled with its ground-truth label.

Obviously, the model isn't particularly brilliant. There is some OK-ish separability of the stair classes which can be identified e.g through DBSCAN. However, while there isn't really any separability of the flat, uphill and downhill classes, I think its still clear there is a kind of gradient-like distribution. Ideally, I'd be looking to try and construct a cluster model looking something like this:

My attempts so far to construct cluster models using K-Means or GMM has resulted in some shockingly poor NMIs (ranging from 10-20%) that resemble nothing like the "ideal" distribution. Are there any clustering models out there that could help, or am I ultimately just never going to get anywhere with this?

P.S I am aware a supervised approach is going to be better, but in this research context I'm trying to look at how the data could be handled without presence of Ground Truth Annotation for training.

Answer 1

Some of the language you're using suggests a support vector machine / support vector classifier. Have you tried that?

You might also try hierarchical clustering, a useful exploratory technique (e.g. with the hclust() function in R). Hierarchical clustering is a multi-step process where each observation starts out as its own "cluster", and at each step, a pair of clusters is merged based on some linkage criterion (you can choose different linkage criteria). This is repeated until finally all clusters will be merged together. We thus end up with a hierarchy of clustering results ranging from $n$ clusters, $n-1$ clusters, $n-2$ clusters, $\dots $ , 1 cluster. This is often visualized as dendrogram, which is a tree-like plot displaying which points/clusters are being merged at each step. This can tell us at various stages which clusters the method/algorithm is able to distinguish or not.

However, for your dataset, it might be too much to ask for any clustering method to perform well. For example, try clustering with a Gaussian mixture model (see e.g. the mclust R package) using the true labels as the initial values. Then compare the output estimated labels with the true labels. If the error rate is very high, then we can hardly expect good clustering results (at least for this class of clustering method).