9
$\begingroup$

Significance tests are used for a variety of reasons and in many research scenarios. One of them is checking the equality of sample sizes.

There's a book for psychologists whose authors recommend chi-squared test to check if two samples are of the same size. First sample size is 50 and the other one is 53. Then they use chi-squared test to find out if 50 is statistically equal to 53. If it is, then a researcher can state equal sample sizes (f.e. for t-tests or ANOVA, etc).

Q: Isn't it a really bad way to use significance tests?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ @kjetilbhalvorsen I'm taking your question as a recognition for my English ;) The book is all in Polish. However, do you think that's bad advise? $\endgroup$
    – Lil'Lobster
    Commented Nov 10, 2020 at 12:58
  • 2
    $\begingroup$ Is this test really usefull ? You can do T-tests or ANOVA with different sample sizes, it's a bit more complex, but using R, it's alright. It does seem to be a poor use of testing. $\endgroup$
    – Pohoua
    Commented Nov 10, 2020 at 13:03
  • 1
    $\begingroup$ Still, can you reference the book (and page number)? I speak Polish and I'm really curoius to see how authors justify their recommendations... $\endgroup$
    – Łukasz Deryło
    Commented Nov 10, 2020 at 14:12
  • 1
    $\begingroup$ "Drogowskaz statystyczny" (2007), p.183. $\endgroup$
    – Lil'Lobster
    Commented Nov 10, 2020 at 14:38
  • 1
    $\begingroup$ It makes no sense whatsoever. It makes sense to test for the prevalence of groups in the population, not to test in-sample sizes. In-sample, if one group is sized 50 and the other 53, then you already know, with 100% certainty, that both samples are of different sizes. $\endgroup$
    – Firebug
    Commented Nov 30, 2020 at 12:50

2 Answers 2

Reset to default
8
$\begingroup$

It does not look like good advice, but not knowing the authors' intention what to say? But, more important, it is not needed. Anova or t-tests with unequal sample sizes is not a problem (it might be inefficient, so in the planning phase try to avoid it.)

See for instance Are unequal groups a problem for one-way ANOVA? and many similar posts you can find by searching this site!

Improve this answer
$\endgroup$
6
$\begingroup$

OK, I got relevant excerpt of the book from OP. It says that ANOVA needs equal smaple sizes across groups and chi-squared test may be used to check this assumption.

Of course, ANOVA doesn't need equal samples, which is nicely explained in link provided by kjetil b halvorsen.

But even if equal sample sizes were needed, testing this assumption makes no sense. It is because statistical test are trying to use information provided in sample from some population to say something about whole population. T-test, for example, tries to locate population mean given a sample from this population. Population mean is unknown and that is why we need testing procedure to tell something about it with, say, 95% certaininty. Notice now, that sample mean is very different from population mean. Sample mean is perfectly known so we just know where it is (with 100% certainity). So we know for sure if it differs from, say, 153.32. Problem is that sample mean is not very interesting quantity, it's population mean that is interesting.

If equal sample sizes were needed we would be, contrary to common use of statistical tests, interested in some quantity (number of elements in this case) in sample, not in population. So we do not need any testing procedure. We can just count observations in our sample and can be 100% sure if it differs from what authors of the book call "ANOVA assumptions".

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.