2
$\begingroup$

How would I go about comparing two populations (A and B) with unequal sample sizes.

Sample size of A, SA: 10,000 Sample Size of B, SB: 2000

My goals are:

  1. To use data visualization tools to study relationships between the variables of each sample (Example: bar graph, histograms, scatter plots)

  2. To compare the means of both the samples.

    a. By hypothesis testing

    b. By simply estimating the mean of each sample such that the sample size of A and B are equal (Example: The sample size of A and B are equal (say 2000) with a mean of 40 and 35 respectively).

Since the sample sizes are very different, I do not know:

  1. a. If I must be worried about such a huge difference in sample sizes (I have not checked but seems like the variances are unequal too).

    b. Is sampling or weighting recommended?

    By sampling I mean: randomly selecting 2000 observations from SA

    By weighting I mean: Assign a constant to the data to increase emphasis of the information from SB.

  2. If there are statistical tests robust to such a huge difference in sample sizes

  3. Do data visualization tools (like scatter plots and bar graphs) render reliable results when sample sizes are highly unequal?

$\endgroup$
1
$\begingroup$

As you want to compare means, let's say you are about to use t-test (there is a normality assumption to test before that).

If I must be worried about such a huge difference in sample sizes (I have not checked but seems like the variances are unequal too).

Don't worry that much about unequal sample size. As far as I know this is not a condition for most of statistical tests. Unequal variances can be a problem for some tests but not for all. There are different t-test formula. The default one in r does assume unequal variance.

Is sampling or weighting recommended?

Not needed. This question seems to imply that you can only compare equal groups, this is not the case, have a look on the welch t-test (the one implemented by default in r). $$ t \quad = \quad {\; \overline{X}_1 - \overline{X}_2 \; \over \sqrt{ \; {s_1^2 \over N_1} \; + \; {s_2^2 \over N_2} \quad }}\,$$ You can see that :

  • Neither sample size or variances are assumed equal.
  • If N1 or N2 is big, the statistic goes high (which is appreciable)

Do data visualization tools (like scatter plots and bar graphs) render reliable results when sample sizes are highly unequal?

No problem either.

Intuitively, I would just point out that the unequal sample size can result in a loss of statistical power compared to more equilibrated samples. 6000-6000 would allow you to see better if there is a significant difference between your groups. But anyway, unless you have a huge variance, 2000-12000 should give you very reliable results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.