0
$\begingroup$

I'm trying to check / evaluate how representative a sample is from a dataset.

I'm interested in two things:

  1. Which sample best represents the original dataset
  2. Is the sample a "good enough" representative of the dataset

Example scenario: I have a dataset

original:

> head(mtcars,n=10)
                   mpg cyl  disp  hp drat    wt  qsec vs am gear carb
Mazda RX4         21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4
Mazda RX4 Wag     21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4
Datsun 710        22.8   4 108.0  93 3.85 2.320 18.61  1  1    4    1
Hornet 4 Drive    21.4   6 258.0 110 3.08 3.215 19.44  1  0    3    1
Hornet Sportabout 18.7   8 360.0 175 3.15 3.440 17.02  0  0    3    2
Valiant           18.1   6 225.0 105 2.76 3.460 20.22  1  0    3    1
Duster 360        14.3   8 360.0 245 3.21 3.570 15.84  0  0    3    4
Merc 240D         24.4   4 146.7  62 3.69 3.190 20.00  1  0    4    2
Merc 230          22.8   4 140.8  95 3.92 3.150 22.90  1  0    4    2
Merc 280          19.2   6 167.6 123 3.92 3.440 18.30  1  0    4    4

and I take 4 samples, sample1 will have 20% of the records, sample2 will have 40%, sample3 will have 60% and sample4 will have 80% of the original dataset (taken at random).

I want to be able to compare these 4 samples with the original and show how their representativeness changes as the percentage changes, even though, all could be good and bad representatives, the larger samples should have a larger chance of being representative (ie I will run the test multiple times).

I have tried using RMSE, MASE, Chi Sq test and a couple of other methods in R but with no luck.

Any and all help will be greatly appreciated!

$\endgroup$
0
$\begingroup$

If you're OK with the assumption that the population roughly follows a multivariate normal distribution, you could do the following:

  1. Estimate the parameters of the distribution from your sample (mean and covariance matrix)
  2. Compute the likelihood of the population data based on those parameters (multivariate normal probability density function)
  3. Compare the population's likelihood based on parameters from different sample sizes
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.