2
$\begingroup$

I'm interested in building an unsupervised clustering model to look at the physical activity data of a person. Below is an image of the model in the 2D space (built through t-SNE), where each data point has been labelled with its ground-truth label. enter image description here

Obviously, the model isn't particularly brilliant. There is some OK-ish separability of the stair classes which can be identified e.g through DBSCAN. However, while there isn't really any separability of the flat, uphill and downhill classes, I think its still clear there is a kind of gradient-like distribution. Ideally, I'd be looking to try and construct a cluster model looking something like this:

enter image description here

My attempts so far to construct cluster models using K-Means or GMM has resulted in some shockingly poor NMIs (ranging from 10-20%) that resemble nothing like the "ideal" distribution. Are there any clustering models out there that could help, or am I ultimately just never going to get anywhere with this?

P.S I am aware a supervised approach is going to be better, but in this research context I'm trying to look at how the data could be handled without presence of Ground Truth Annotation for training.

$\endgroup$

1 Answer 1

1
$\begingroup$

Some of the language you're using suggests a support vector machine / support vector classifier. Have you tried that?

You might also try hierarchical clustering, a useful exploratory technique (e.g. with the hclust() function in R). Hierarchical clustering is a multi-step process where each observation starts out as its own "cluster", and at each step, a pair of clusters is merged based on some linkage criterion (you can choose different linkage criteria). This is repeated until finally all clusters will be merged together. We thus end up with a hierarchy of clustering results ranging from $n$ clusters, $n-1$ clusters, $n-2$ clusters, $\dots $ , 1 cluster. This is often visualized as dendrogram, which is a tree-like plot displaying which points/clusters are being merged at each step. This can tell us at various stages which clusters the method/algorithm is able to distinguish or not.

However, for your dataset, it might be too much to ask for any clustering method to perform well. For example, try clustering with a Gaussian mixture model (see e.g. the mclust R package) using the true labels as the initial values. Then compare the output estimated labels with the true labels. If the error rate is very high, then we can hardly expect good clustering results (at least for this class of clustering method).

$\endgroup$
3
  • $\begingroup$ Hi iceberg, thanks for your response re: Support Vector Machine, as far as I know, SVMs are always supervised and use knowledge of the label to help construct the boundaries. As I said in the P.S, I'm deliberately not using SVM since I'm trying not to use Ground Truth to help construct clusters (or more accurately, I have done supervised learning and this is to supplement/contrast that research). But if thats not true and you can use SVM in an unsupervised context I'm all ears. I omitted that I've also used Hierarchical, but to no avail (similar performances to KMeans and GMM) $\endgroup$
    – ZChemZ
    Jul 7, 2021 at 21:15
  • 1
    $\begingroup$ @ZChemZ Oh yes, you are correct about SVM. Another method occurred to me that you could try is convex clustering, as shown in the paper, "Splitting Methods for Convex Clustering," from Chi and Lange, freely available through the arXiv system. The authors also wrote an R package for it, cvxclustr. Take a look at the k-nearest neighbors weighting scheme for their convex clustering. $\endgroup$
    – iceberg
    Jul 8, 2021 at 16:47
  • $\begingroup$ I'll give that convex clustering a look into, thanks so much. $\endgroup$
    – ZChemZ
    Jul 10, 2021 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.