I asked a number of 71 'experts' to sort 92 different psychological constructs based on their similarity. Based on their answers, I constructed a dissimilarity matrix.
Initially, I wanted to analyse the data using hierarchical cluster analysis (HCA). Eventually, however, I decided to go for partitioning around medoids (PAM). The main reason has been that PAM allows me to identify a representative example for each cluster determined. It basically allows me to label those clusters by the means of these examples. In my next phase of my research, I want to question these experts on these representative examples.
I run PAM in R using the
cluster package. As for the number of clusters, I created a loop testing the solution for each number of clusters from $k=2...91$. Using the average silhouette width as the benchmark, I determined the optimal number of clusters:
As one may be able to see in the graph, the optimal cluster number is $2$ (I also confirmed this using
fpc package. I am somewhat "disappointed" about the optimal number of 2 clusters. Since I plan to interview experts about each of the clusters (what is it? what are the effects etc.? I hoped to get a larger number of clusters.
How common and "justifiable" is it to deviate from optimal cluster number that has been statistically derived for PAM applications? For example, around 43 there is another "elbow". If I decided to go for this, to what extent would this be unfounded?