3
$\begingroup$

So basically, we have a sample of known data taken from a larger sample of known data. Is there a good way to measure how "representative" that data is, especially in distributional terms, compared to the larger sample from which it was taken? I'm basically looking for a specific metric that compares their distributions, and I would theoretically add data from the larger sample into the smaller and the metric should say there is an improvement.

It seems like the Kolmogorov–Smirnov test is what I am looking for more-or-less.

The use case is if I want a small representative sample from a large store of data to shove in a model without hold-out testing to see how that model generalizes--yes that's bad but that's this specific use case. Because of this limitation, I have to know that the smaller sample I want to put in the model is representative of the larger dataset.

$\endgroup$
2

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.