0
$\begingroup$

I am studying geographic solar radiation data obtained from satellite images. I would like to and use cluster analysis and make some correlation analysis by clusters, instead of by pixel - as some authors have done.

The kind of data is not much 'clusterable', so the main idea here is just to reduce dimensions. I would like to use the PAM algorithm and, to decide the number of clusters, for that I would like to verify how much loss of information would I lose by doing so. I thought in using information from the objective function (i.e. sum of dissimilarities from a point to the medoid of its cluster).

How could I find the explained variance using k-medoids or PAM? Or which would be a good metric to show how much information is lost (or kept) when clusterizing?

$\endgroup$
2
  • $\begingroup$ Sounds like you want to do PCA, which does both dimensionality reduction and you get a result of explained variance for each principal component. $\endgroup$ Oct 19 '18 at 10:27
  • $\begingroup$ Kind of. The advantages of using the cluster would be to assign each element to a single set (which would be together with their neighbors), and then, to compare each set. The dimension reduction I would like is on the space of elements, not on their's variables. $\endgroup$ Oct 19 '18 at 17:14
2
$\begingroup$

Explained Variance is a natural fit for both k-means and PCA, because both methods perform a least squares optimization. It's easiest to see with k-means, which in every step reduces within-cluster variance - until convergence.

PAM doesn't. Not unless you'd use PAM with squared Euclidean, at which point it will still only find a worse solution than k-means, and slower.

So while it's straightforward for R^d data how to compute "variance explained" with PAM (total variance of all data - sum of total variances of each cluster) this metric is not an appropriate choice for PAM.

You can, however, interpret variance explained as the quantity $$\frac{\sum_x\sum_y d(r(x),r(y))^2}{\sum_x\sum_y d(x,y)^2}$$ With or without the square, where r(x) the "reconstructed position" or "replacement" of x. Then this measures - to some extend, assuming that you are required to reduce distances overall, as it would be trivial to get arbitrarily large values - how well you approximate the data when replacing each point x with r(x). With PAM, the non-squared version seems most appropriate. It would be more meaningful to rather use some form of a squared normalized loss in reconstructing the pairwise distances, e.g., $$\frac{1}{N^2}\sum_i\sum_j \left(\frac{d(x,y)-d(r(x),r(y))}{d(x,y)}\right)^2$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.