Best test to use for investigating if the difference between means is significant?

Question

I'm studying the length of stay for patients in a hospital. I have a sample of n=4533 observations. Each of these observations are assigned to an admin group numbered between 1 and 8, based on the reason they were admitted to hospital. Admin group 2 has the characteristics:

n = 193, x̄ = 37.2020725 (days), s.d. = 35.6247163 (days)

This is the highest mean of the 8 admin groups. I want to test whether the difference between the other groups means and this mean is significant. If I combine the other 7 admin groups, I get the characteristics:

n = 4340, x̄ = 25.5078341, s.d. = 31.1011062

I tried to run a t-test to compare these 2 sets of data, but I ended up getting really small values for standard error and degrees of freedom (less than 1). I'm assuming the t-test is inappropriate for this data, perhaps due to the fact that 1 sample size is significantly larger than the other.

Can anyone think of a suitable test to help with what I'm trying to investigate here? Alternatively, should I change my angle and try an ANOVA test (if that is appropriate?), to study whether admin group 2's mean is significantly different from all the other groups' respective means? Hope I made my question clear.

Answer 1

There is a Bayesian equivalent of the t-test. It returns the actual probability that the means are different. You can also use different sample sizes for each distribution.

You can find the paper here: https://jkkweb.sitehost.iu.edu/BEST/

An online demo can be found here: http://www.sumsar.net/best_online/

Answer 2

Minitab statistical software will accept summarized data, such as you provide. Assuming that data are sufficiently nearly normal and admission groups have nearly equal variances, the following output from Minitab for Welch 2-sample, 2-sided t test might be useful:

Two-Sample T-Test and CI 

Sample     N  Mean  StDev  SE Mean
1        193  37.2   35.6      2.6
2       4340  25.5   31.1     0.47

Difference = μ (1) - μ (2)
Estimate for difference:  11.70
95% CI for difference:  (6.56, 16.84)
T-Test of difference = 0 (vs ≠): T-Value = 4.49  P-Value = 0.000  DF = 205

Notice that SE Mean for the larger (combined) group is very small. Thus, in effect, your two-sample test is not much different from a one sample test on the first group of $H_0: \mu = 25.5$ against $H_a: \mu \ne 25.5.$ As follows:

One-Sample T 

Test of μ = 25.5 vs ≠ 25.5

  N   Mean  StDev  SE Mean      95% CI         T      P
193  37.20  35.62     2.56  (32.14, 42.26)  4.56  0.000

@Knarpie is Correct that testing one level of a factor against the others is controversial. However, one can imagine that there is no harm in it here. If you had done a one-factor ANOVA comparing all factors, then is seems that ANOVA would have had a very small P-value, and your two-sample t test is not much different from looking ad hoc at the 'contrast' that compares one level against the others. With 8 levels of the factor one might look at as many as ${8\choose 2} = 28$ paired comparisons ad hoc. Even the overly conservative Bonferroni protection against 'false discovery' would consider P-values below 5% as significant.

If you are writing a formal report, I suggest you begin with an appropriate one-way ANOVA, check to see whether residuals are consistent with normal data and levels have about the same variances, and (if so) do the appropriate ad hoc comparisons. Then report the significant findings that are of practical importance.