# Test two groups with only sample statistics or without distributional assumptions

I have two sets of samples, A and B. I want to find whether the underlying mean (i.e. if the sample size was infinite) of A is greater than that of B, to a certain confidence (95%).

There are two scenarios for this:

1. All I know is the mean, standard deviation and sample size for both. For A, these are (0.73, 1.34, 30), and for B, these are (0.67, 1.21, 30). I then make the assumption that samples are normally-distributed.

2. I have the values for all the individual samples, and make no assumption about the underlying distribution.

How can I do the hypothesis test for each of these two scenarios?

1) You'd use a t-test. In this case sample sizes are identical, so there's not even a need to worry about Welch adjustment; the ordinary t-test would do.

2) That depends; if you're prepared to assume identical distributions up to a location shift, then if population means are finite you could certainly use a Wilcoxon-Mann-Whitney for that. However, it's considerably more broadly applicable for a test of equality of means than that*. [One advantage of restriction to a location-shift alternative is interpretability, but it's not necessary for the test to be suitable. Scale shift alternatives - and a host of other transformations that imply ordered means - would all be included]

If you assume only identical distributions under the null (no need to restrict it to location shift alternative or even to a stochastic-dominance alternative), you could do a permutation test on the difference in means. (There are still other options though)

What software are you using?

* Here's an illustration that it can be broader and still be about means.

(i) Assume $X_i$, $Y_j$ are both continuous from $F_X$ and $F_Y$ respectively (each with finite mean), with the usual independence assumptions and assume for this argument that both variables are non-negative. Let $S_X=1-F_X$. Let's take our hypotheses to be

$H_0: F_X=F_Y$ vs $H_1: F_X<F_Y$

Then that's equivalent to

$H_0: S_X=S_Y$ vs $H_1: S_X>S_Y$

Integrate the expressions on both sides on the positive half line and we get:

$H_0: E(X)=E(Y)$ vs $H_1: E(X)>E(Y)$

as long as $\int_0^\infty S_X(t)-S_Y(t) dt >0$ and we can interchange the integration and the difference.

[This sort of argument can be easily extended to a situation with a lower bound on the variables other than zero.]

(ii) the restriction to positive random variables was a convenience in order to be able to bring in the argument about the relation between survival functions and expectation; it's not necessary. For example, second order stochastic dominance implies the expectations are ordered.

I'm not currently clear on what the weakest restrictions (on top of the assumptions already made for the WMW) above first order stochastic dominance would need to be to imply ordering of expectations, but it's whatever minimum assumption is needed for $F_X-F_Y<0$ in some region to imply $E(X)-E(Y)>0$. (I think if the inequality $F_X-F_Y\leq 0$ is somewhere strict in such a way that their integrals differ in that region, it should be possible to show that $E(X)-E(Y)>0$ follows.)

I hope to come back to this and say some more once I've thought about it (and maybe learned some more). The implication is that the Wilcoxon-Mann-Whitney is useful for testing means under much less restrictive assumptions than are often suggested.

As gung points out, we still need some assumptions. The Wilcoxon-Mann-Whitney is sensitive to an even broader set of alternatives than I'm discussing here, so without additional assumptions, rejection doesn't necessarily suggest a difference in means.

• Is your recommendation against the MW when the distributions differ b/c the MW can be significant w/ equal means but different shapes? Commented Mar 30, 2015 at 16:11
• @gung While your comment is true (and a consideration to keep in mind), I didn't recommend against W-MW anywhere in my answer. I offered an alternative which may be useful. In fact with some care, one can extend the applicability of the W-MW from the equal shape location-shift case (e.g. if both means exist, we could extend it to the fairly general situation under the alternative of stochastic dominance, since it implies unequal mean). The W-MW is ... a very handy test. If you face heavy tails, it should have better power than the permutation test. Commented Mar 30, 2015 at 21:13
• Hmm, maybe I'm misunderstanding. Can't you have 2 distributions, 1 positively skewed & 1 negatively skewed, but w/ identical means, st NS is stochastically dominant over PS & hence MW would be significant w/ identical means? I thought that was the case, & I read your answer as suggesting MW only w/ identical shape b/c of that potential problem. I may be confused / misunderstanding something. Commented Mar 30, 2015 at 21:36
• @gung I suspect that I was being unclear. I mean stochastic dominance at first order; $X$ stochastically dominates $Y$ at first order if $F_Y\geq F_X$ (I should have started the CDF relationship). Because of the relation between expectation and the survival function ($1-F$), if that inequality is strict (as under the alternative), or even strict in some interval and elsewhere equal, such that the integral of the survival functions differ, that implies $E(X) > E(Y)$. Commented Mar 30, 2015 at 21:45
• er, that should read "... started with the CDF..." and the "(as under the alternative)" should be later in that last sentence, since it includes the second case. For anyone reading along, the relationship between survival function and expectation is discussed here Commented Mar 30, 2015 at 21:51

To clarify on @Glen_b's first point, we typically think of calculating the $t$-test by feeding the data into some software (the wide availability of advanced, easy to use software has spoiled people, the kids today...). But in the old days, we ran $t$-tests by hand (after walking twelve miles uphill through the driving snow), and to do that you would first calculate the means and SDs, and then enter those into the formula.
$$t = \frac{\bar x_2 - \bar x_1 - (\rm null\ difference)}{\sqrt{\frac{s_p^2}{n_1}+\frac{s_p^2}{n_2}}}$$ where, $$s_p = \sqrt{\frac{(n_1-1)s_1 + (n_2-1)s_2}{n_1+n_2-2}}$$ If you want to use software, there are often options that let you input sample statistics instead of data. For example, in R you could use ?tsum.test in the BSDA package.

library(BSDA)
tsum.test(mean.x=.73, s.x=1.34, n.x=30,
mean.y=.67, s.y=1.21, n.y=30, alternative="greater", var.equal=TRUE)
#         Standard Two-Sample t-Test
#
# data:  Summarized x and y
# t = 0.182, df = 58, p-value = 0.4281
# alternative hypothesis: true difference in means is greater than 0
# 95 percent confidence interval:
#  -0.4909958         NA
# sample estimates:
# mean of x mean of y
#      0.73      0.67


Note that you could not combine your two scenarios. That is, if you only have sample statistics, you must assume that the data are normally distributed to perform a $t$-test.