I tried to cluster my data using spectral clustering algorithm. Before applying clustering algorithm, I used PCA on the data, which gave me 4 PC accounting for 95% of variation. After that I plotted eigenvalues for the clusters for two datasets (as I need to analyse them separately). From the plots it's not so obvious that 4 clusters is the right choice (index is a number of clusters). What would be your recommendation for choosing the number of clusters? Also, what is the best approach in choosing number of neighbors considered?Thank you.
1 Answer
$\begingroup$
$\endgroup$
You said you're doing spectral clustering. This means you're constructing a similarity matrix, a graph Laplacean, etc. If you don't see the sharp drop in eigenvalues at 4, then maybe you should alter the parameters of your similarity matrix (i.e. if it's a Gassian kernel, alter the bandwidth; if it's KNN, alter the k, etc).