Is there a way to calculate the Explained Variance with PAM or k-medoids?

Question

I am studying geographic solar radiation data obtained from satellite images. I would like to and use cluster analysis and make some correlation analysis by clusters, instead of by pixel - as some authors have done.

The kind of data is not much 'clusterable', so the main idea here is just to reduce dimensions. I would like to use the PAM algorithm and, to decide the number of clusters, for that I would like to verify how much loss of information would I lose by doing so. I thought in using information from the objective function (i.e. sum of dissimilarities from a point to the medoid of its cluster).

How could I find the explained variance using k-medoids or PAM? Or which would be a good metric to show how much information is lost (or kept) when clusterizing?

Answer 1

Explained Variance is a natural fit for both k-means and PCA, because both methods perform a least squares optimization. It's easiest to see with k-means, which in every step reduces within-cluster variance - until convergence.

PAM doesn't. Not unless you'd use PAM with squared Euclidean, at which point it will still only find a worse solution than k-means, and slower.

So while it's straightforward for R^d data how to compute "variance explained" with PAM (total variance of all data - sum of total variances of each cluster) this metric is not an appropriate choice for PAM.

You can, however, interpret variance explained as the quantity $$\frac{\sum_x\sum_y d(r(x),r(y))^2}{\sum_x\sum_y d(x,y)^2}$$ With or without the square, where r(x) the "reconstructed position" or "replacement" of x. Then this measures - to some extend, assuming that you are required to reduce distances overall, as it would be trivial to get arbitrarily large values - how well you approximate the data when replacing each point x with r(x). With PAM, the non-squared version seems most appropriate. It would be more meaningful to rather use some form of a squared normalized loss in reconstructing the pairwise distances, e.g., $$\frac{1}{N^2}\sum_i\sum_j \left(\frac{d(x,y)-d(r(x),r(y))}{d(x,y)}\right)^2$$