1
$\begingroup$

If I am comparing means of say four groups with unequal variances, I will have to go pair wise. So it would be 4C2 combinations.Now I can get different pair with mean significantly different, Is there any way we can claim that this specific group's mean is significantly different than all others? Suppose the groups are G1,G2,G3,G4.

Diff of G1,G2 : Significant

Diff of G2,G3: Significant

Diff of G1,G4: not Significant

Diff of G1,,G3: not Significant

.................................

....all other comb insignificant... ................... ...................

Is there any way to reach at conclusion saying mean of G3 is significantly different from all others and G3 is the main culprit?

$\endgroup$

2 Answers 2

2
$\begingroup$

You can use Welch t-tests for the pairwise comparisons, and something like Bonferoni's correction or Tukey's HSD to correct for multiple comparisons (alternatively, you could not correct it --- the choice would depend on if you're more concerned about Type 1 error or Type 2 error). If this is an exploratory analysis --- i.e. you don't have any particular hypothesis about which groups should differ --- then you should test all pairwise comparisons (using something like Tukey's HSD) and be sure to get descriptive statistics on the groups (mean and SD, probably also min, max, and anything else that might be relevant) so you have some context for interpreting the pattern of results.

If you want to directly test the hypothesis that G3 differs from the other groups and they do not differ from each other, then planned contrasts, such as Helmert contrasts are a more elegant solution. In this case, you would test the following three contrasts (or something like it):

G3 vs. mean(G1, G2, G4)
G4 vs. mean(G1, G2)
G2 vs. G1

If you were to see a significant difference for the first contrast but not the second two, you might conclude that G3 is the "culprit" (although this kind of interpretation is common, note the danger of interpreting a non-sig difference as "no difference", though, especially in underpowered designs!).

Note that in the specific example you give, though, I personally don't see evidence that G3 is different from all of the rest. G1 and G2 are significantly different, and G3 does not differ from G1. If that's your hypothesis, it looks like it's not cleanly supported by the data. If you haven't already, I recommend using a boxplot to visualize the differences among the groups, since it appears your expectations don't quite line up with the data.

$\endgroup$
5
  • 1
    $\begingroup$ Tukey's HSD seems relevant here as well. $\endgroup$
    – AdamO
    Feb 13, 2018 at 19:22
  • $\begingroup$ Many Thanks Rose,used Welch t-tests for the comparison. I'v data of persons with number of training days & I'v four independent group with this data.I'm trying to find whose average training days is significantly different. Definitely the mean would give first hand impression but validating it using Welch t-test.I don't know which group is on higher end so trying to needle out top groups with significant average training days. I'l use it for further analysis in tht specific group.Pls correct me if my approach is wrong.Thanks again. $\endgroup$
    – vaidehi S
    Feb 14, 2018 at 6:20
  • $\begingroup$ As you have suggested above, will it be a good approach to apply Welch T-test forming pair as below: ---> mean G1 vs. mean(G2, G3, G4) --->mean G2 vs. mean(G1, G3, G4) --->mean G3 vs. mean(G1, G2, G4) --->mean G4 vs. mean(G1, G2, G3) Which ever results in significant difference I will then take those groups only for my further analysis. Apologies if I am ambiguous. $\endgroup$
    – vaidehi S
    Feb 14, 2018 at 6:56
  • $\begingroup$ @AdamO Good call. I've edited my answer a bit to clarify that a Bonferroni correction is one option among several for the multiple comparisons issue. $\endgroup$ Feb 14, 2018 at 16:42
  • $\begingroup$ @vaidehiS I'm not sure I understand your comment (perhaps edit the question if you want to add more details about your planned analysis?), but from what I can tell it's not what I'm suggesting. The comparisons you specify are not Helmert contrasts, and the logic is quite different from what I suggested. Also: "I don't know which group is on higher end" -> then this should all be exploratory. Use Tukey's HSD to get all pairwise comparisons, and a boxplot to visualize. $\endgroup$ Feb 14, 2018 at 16:45
1
$\begingroup$

If you don't mind relaxing your null from mean difference to stochastic size, the Kruskal-Wallis test is a nonparametric analog to the oneway ANOVA, which does not require the assumption of equal variances (nor does it require assumptions about the distributions beyond finite and i.i.d.). As the rank sum test is to the unpaired t test, so the Kruskal-Wallis is to the oneway ANOVA.

The null hypothesis of the Kruskal-Wallis test is:

$H_{0}: P(X_{i} > X_{j}) = 0.5$ for all $i \ne j$, where $i$ and $j$ identify your four groups), and
$H_{0}: P(X_{i} > X_{j}) \ne 0.5$

Post hoc pairwise tests following a rejection of the Kruskal-Wallis omnibus hull hypothesis, include Dunn's test and the less well-known Conover-Iman test.

References

Kruskal, W. H. and Wallis, A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260):583–621.

Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.

Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.