Determine clusters for the encodings of a Siamese Neural Network

Question

I implemented a Siamese Neural Network that encodes images of different objects and outputs "coordinates" for each image in a lower dimension. My goal is to measure how good the network is clustering the images.

If I visualize the coordinates in two to three dimensions (by applying PCA) I can clearly see that images of the same object are close to each other. The algorithm works well enough. Therefore, I am defining the clusters based on the ground truth labels, which is obviously not the best approach.

So my question is how should I determine the real clusters and also what would be the best measure to determine how "good" those clusters are?

I thought about running a clustering algorithm (such as k-Means) on top of the encoding and then use an external validation metric such as the F-Score to measure the goodness of the overall model.

Answer 1

There are two approaches; direct methods and statistical testing methods:

Direct methods: consists of optimizing a criterion, such as the within-cluster sums of squares or the average silhouette. The corresponding methods are named elbow and silhouette methods, respectively.

Statistical testing methods: consists of comparing evidence against the null hypothesis. An example is the gap statistic.

1. Elbow method - see this paper for details. Essentially, a line is plotted on a graph and the bend in the line indicates a possible number of clusters. In R it can be done using the NBClust package.

2. Average silhouette method computes the average silhouette of observations for different values of k. The optimal number of clusters k is the one that maximizes the average silhouette over a range of possible values for k (Kaufman and Rousseeuw 1990).

3. The gap statistic compares the total within intra-cluster variation for different values of k with their expected values under the null reference distribution of the data. The estimate of the optimal clusters will be a value that maximizes the gap statistic (i.e, that yields the largest gap statistic). This means that the clustering structure is far away from the random uniform distribution of points.

The fviz_nbclust() function `in factoextra R package can be used to compute the three different methods elbow, silhouette and gap statistic for any partitioning clustering methods like K-means, K-medoids (PAM), CLARA, HCUT.

Besides, you can also look at some similar questions asked in 1, 2

For cluster validation metrics, see this post

Finally, remember clustering is an explorative technique. Assuming "true" clusters is a paradox. Your objective should be in exploring different clusterings of the same data to learn more about it. Treating clustering as a black box never works well.