# Are there any quantitative metrics for how representative a sample is?

I'm interested in selecting a sample that is representative of a population. Additionally, I want to be able to quantitatively measure the representativeness of a sample. For example, is there a way to determine, for a sample size n, where n is some fraction of the population size x, how representative the sample is? Can I determine if a given sample is the most representative of a population? What if I introduce weighting of importance for each property? What if the desired representative sample size is 2n or n/2? How would this change the selection?

Are there any resources (textbooks, literature, etc...) that would lead to some relevant statistical measures/tests to answer such questions?

• Since (except in trivial cases) it is impossible for any proper sample to be the same as the population and in any case there are infinitely ways in which one could compare a sample to the population, it is incumbent upon you to stipulate exactly how you would make that comparison: it's not a question that has a universal answer. Perhaps the most relevant literature would be the initial discussion of random sampling in any really good elementary stats textbook, such as Freedman, Pisani, & Purves, Statistics. – whuber Jan 30 '17 at 22:26
• If you want to see how a sample represents the population, it seems like you'd need at least 3 things--a set of traits that you want 'represented', some knowledge of how those traits exist in the population, and a mathematical definition of what it means to be 'represented well'. The first one is easy to get but possibly difficult to pin down in a way that helps you learn the right thing about the sample, the second you could guess at from other, larger samples (which might have its own problems), and the third seems like the same pitfalls as the first. – Sullysaurus Jan 30 '17 at 22:28
• @Sullysaurus I actually have the first 2 of your stated criteria. I'm working with materials so I have a set of properties I'm interested in, and these properties are databased for all materials. I just need to find a metric for 'represented well.' Would you have any suggestions in developing one, or might some definitions already exist? – Tunk Jan 31 '17 at 0:49
• As other commenters mention, there are many ways to do this. Here's just one example of a class of ideas that might help getting started. Consider the empirical distribution of your sample, and compare this to the distribution it was sampled from. To make this comparison, check out the Kolmogorov-Smirnov, Cramer-von Mises, and Anderson-Darling tests. Also check out Kullback-Leibler divergence, cross entropy, and various statistical distance measures (en.wikipedia.org/wiki/Statistical_distance) – user20160 Jan 31 '17 at 2:49