Assume a set of ten simulated frequency distributions that are contingent on an independent variable that reflects two treatment groups: Nine of the distributions represent treatment group 1 (in blue in Fig.1), while the tenth represents treatment group 2 (in red in Fig.1). All ten distributions have the same sample size, but none is normally distributed.
Visually I confirm that mean and median of the single frequency distribution of treatment group 2 is strongly different than means and medians of the other distributions, which do display slight variation among each other (Fig.1).
To evaluate the omnibus H0, I perform a Kruskal-Wallis test, which allows me to reject the omnibus H0. To conduct individual comparisons between the distributions, I perform pairwise Kruskal-Wallis tests in the post-hoc framework of Nemenyi. These post-hoc tests indicated rejection of H0 between distribution of treatment group 2 to any of the other distributions. However, H0 can also be rejected in some of the comparisons within treatment group 1, indicating the presence of another explanatory variable.
What strategy (e.g., test design, statistical framework, etc.) would you employ to compensate for this second, unrecognized explanatory variable. All I am interested in is to test H0: distributions [plural!] of treatment group 1 identical to distribution of treatment group 2.