I have recently sent a paper on aflatoxin levels in nuts for publishing. The aim of the study was to analyze a total amount of 106 nuts. The nuts were divided into four categories: walnuts (n=33), hazelnuts (n=19), pistacchio (n=29) and almonds (n=25). The reviewer seems not to have a broad knowledge on study design and statistical analysis, which can be seen in his critique: "In my opinion, every sample should be equal in size, in order to improve the quality of the research." I totally disagree with his opinion, mostly because following statistical analysis did not require the same sample size. This was a cohort epidemiological study, not a experimental one. Can somebody back me up with references or explain in details why do i don't need equal sample sizes to do a high quality research?

Or in other words: Would the research be better if i compared 4 categories with the same sample size (eg. walnuts=23, hazelnuts=23, almonds=23, pistacchio=23), compared to my sample sizes?

  • $\begingroup$ You don't need equal sample sizes but having equal sample sizes can increase the power of your tests. $\endgroup$ – StatsStudent Feb 9 '15 at 19:42
  • $\begingroup$ Was the result 'significant'? Or are you trying to assert the null here? $\endgroup$ – gung Feb 9 '15 at 19:57
  • $\begingroup$ I've posted a more detailed answer down there with the results :) $\endgroup$ – Ivn Bubrov Feb 9 '15 at 20:11

I feel that your observational study design argument is strong. There is no way to trim all the group down to 19; that would be silly.

Even or uneven group assignment is not the main issue, but rather your samples are large enough to detect a meaningful difference in a group as small as 19. This is also known as the statistical "power" of the sample. If you attached a power calculation, and support with an argument that even your smallest group can help detecting a meaningful difference, then there shouldn't be a point to criticize the uneven distribution.

The reviewer is half-correct. This is perhaps due to some rules of thumb gone wrong. Given all criteria being the same, even groups have higher power than uneven groups. If 15-15 have 80% power to detect a difference of, say, 5 mmol/dl of toxin. Then a 10-20 splitting will become underpowered (e.g. lower than 80%), becoming less likely to detect the same difference if such difference does exist.

  • $\begingroup$ My thoughts exactly, @Penguin_Knight. $\endgroup$ – StatsStudent Feb 9 '15 at 19:44
  • $\begingroup$ To be more precise: The maximum level of aflatoxins in a nut is 4ug/kg. I am interested how many nuts have aflatoxins above this critical value. Luckily only 9 nuts have had >4ug/kg aflatoxins (see the attachment Table 1: Almonds=5, Hazelnuts=3, Pistacchio=1, Walnuts=0). A chi square test was performed to see if there is a statistically significant difference between the nuts (p=0,0225). Due to the 0 walnuts and small sample size of the >4 ug/kg nuts (5,3,1,0), I chose not do a pairwise comparison, but just to list the mean, s.d., etc. Here's the data: filedropper.com/nutdata $\endgroup$ – Ivn Bubrov Feb 9 '15 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.