1
$\begingroup$

I'm comparing 1 continuous variable across 36 groups (nurses grouped into 36 hospital units). Several groups have only 1 observation, but most have more, ranging from 1-24 observations per group. Total observations across all groups are 120.

Here are the results and problems with each of the tests I've done (using SPSS 26):

One-way ANOVA: p = 0.002, but a Levene's test yields p=0.002 based on mean (p=0.121 based on median, not sure if that's relevant) indicates heteroscedasticity, which rules out ANOVA since my group sizes are so different.

Welch's ANOVA: Cannot produce results because some groups have only 1 observation. SPSS states, "Robust tests of equality of means cannot be performed because at least one group has the sum of case weights less than or equal to 1."

Kruskal-Wallis: p = 0.006, but some sources indicate that this test is not valid for data with unequal variances, and I've also read that this test should only be done for groups of sample size >=5.

Is there a test appropriate for comparing these groups? Or can I omit the groups with just 1 observation?

Thanks so much!

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Generally, Kruskal-Wallis makes no assumption about the dispersion, variances, etc between groups. It (and the rank sum test) only require that assumption (along with the distribution shape assumption) if you want to make inferences about differences in mean or median (i.e. location shift). $\endgroup$ – Alexis Jun 21 '19 at 5:00
  • $\begingroup$ Thanks for your help! Isn't location shift what Kruskal-Wallis is measuring? What other inferences can you make from that test that don't require assumption of equal variances? $\endgroup$ – Jay Jun 21 '19 at 5:18
  • 1
    $\begingroup$ Location shift is a very specific use-case which requires extremely stringent assumptions. The KW (and rank sum) is a test for difference in stochastic size ("zeroth order stochastic dominance). The null of KW is "Randomly selected observations from no group are more likely to be larger than any other group", or $H_{0}$: $P(X_{i} > X_{j}) = 0.5$, $H_{A}$: $P(X_{i} > X_{j}) \ne 0.5$ for some $i \ne j$ and $i, j \in [1, 2,\dots ,k]$. $\endgroup$ – Alexis Jun 21 '19 at 6:30
  • 1
    $\begingroup$ That definition of stochastic dominance seems like as meaningful a measure of difference as any to my non-expert brain. Taking your and David's advice, I'll use the Kruskal-Wallis H test and omit the singleton groups. Doing this yields significance of p = 0.004. $\endgroup$ – Jay Jun 22 '19 at 5:48
0
$\begingroup$

Kruskal-Wallis looks like the most reliable option to me, as all other options have one of their major assumptions broken. I would not expect any miracles, though. Tests on small samples will always come with low power, regardless of method used.

I would definitely remove groups with only one observation. More so if variance is expected to be different for every group. If group A shows mean 30 and sample standard deviation 1 and group B consists of only one observation, let's say 35, but we have no clue about its standard deviation, do we say group B has a significantly larger mean than group A? I'd rather say there's no way to tell.

Is it possible that the discrepancy between mean-based tests and median-based tests is due to the presence of outliers in certain groups?

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks! After omitting singleton groups, Kruskal-Wallis H test yields p=0.004, so low power isn't a concern in this case (if I'm understanding the concept of power correctly), I'm mainly worried about presenting a false positive by breaking assumptions of the test I'm using. Regarding outliers--after removing them Levene's test still diverges based on mean (p = 0.008) and median (p = 0.190). It seems like Kruskal-Wallis and omitting singletons is the way to go, please correct me if I'm wrong. $\endgroup$ – Jay Jun 22 '19 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.