Let's say you want to cluster some objects, say documents, or sentences, or images.
On the technical side, you first represent these object somehow so that you could calculate distance between them, and then you feed those representations to some clustering algorithm.
Externally, however, you just want to group similar (in some sense -- and that's where things become pretty vague for me) objects together. For example, in case of sentences we want for clusters to contain sentences about similar topic/concept; we feel that sentences "oh look at this pic of a cute lolcat" and "facebook revealed new shiny feature tonight" should be in different clusters.
What are the usual approaches for measuring this "external" quality of clustering? I.e. we want to measure how well our clustering procedure groups initial objects (sentences, images); we're not interested in internal measures (like averaged cluster radius, clusters sparseness), since those measures deal with objects' representations, not with real objects. Meaning, the chosen representation may be awful, and even if internal measures is great, externally we'll end up with clusters that are complete junk from our vague, subjective, "some sense"-ish point of view.
P.S. Having limited knowledge in clustering domain, I suspect I may be asking about really obvious thing, or my terminology may sound strange to clustering experts. If so, please advice what should I read on the subject.
P.P.S. Just in case, I asked the very same question on Quora: http://www.quora.com/How-to-evaluate-external-quality-of-clustering