Which test to use to compare means/medians when one variable has normal distribution and the other does not?

Which 2-tailed test is best to use to compare means/medians when one variable has a normal distribution and the other does not?

• How do you know one is normal and one is not? How big are your samples? What kind of data are they? What have you looking into? Is this homework? If it is, tag it as such. – John Dec 21 '12 at 4:33
• Can you confirm you mean the mean/median of a variable from two groups which are independent? – Peter Ellis Dec 21 '12 at 5:18

There are whole books - or at least chapters and sections of books - dedicated to just this question, and there is no simple answer. Hence you will get comments asking for more information on what you need and the overall research question and characteristics of the data.

Unfortunately the "mean/median" distinction is non-trivial. Partly because they have a breakdown point of zero (a single extreme outlier makes them completely unrepresentative), means are particularly susceptible to outliers and in general dangerous to estimate from unknown distributions, but most of the traditional techniques are explicitly aimed at means (those are minuses and pluses for using mean as your target parameter).

Sometimes you really need to know the mean - for example, because an official statistics agency need to know the total (mean x population) expenditure from some subgroup, and cannot deal with a general measure of the "average" expenditure such as trimmed mean or median (which is really just a trimmed mean with 50% trimming). In other cases the median or trimmed mean is more useful because you are after a general sense of if a particular group has a higher score "on average" than the others.

Here is an extended quote from the section "Making decisions about which method to use" from Rand Wilcox's Modern Statistics for the Social and Behavioral Sciences, CRC Press. I thoroughly recommend the full book.

Numerous methods have been described for comparing two independent groups. How does one choose which method to use? There is no agreed upon strategy for addressing this issue, but a few comments might help.

First, and perhaps most obvious, consider what you want to know... Despite the many problems with methods for comparing means, it might be that there is explicit interest in the means, as opposed to other measures of location... in this case, the R function yuenbt() (a bootstrap-t method of comparing timmed means provided by Wilcox and available from R-Forge) seems to be a relatively good choice for general use (setting the argument tr=0), with the understanding that all methods for means can result in relatively low power, inaccurate confidence intervals, and unsatisfactory control over the probability of a Type I error....

... If, for example, boxplots indicate that the groups have no outliers and a similar amount of skewness, a reasonable specualtion is that using other measures of location (not the mean) will make little or no difference. But the only way to be sure is to actually compare groups with another measure of location.

If the goal is to maximize power... comparing groups with a 20% trimmed mean is likely to be best, the but the only certainty is that exceptions occur....

... If the goal is to make inferences about a measure of location, without being sensitive to heteroscedasticity and skewness, methods based on a 20% trimmed mean seem best for general use. The bootstrap method for medians also performs well based on this criterion. Generally it is better to use a method that allows heteroscedasticity rather than a method that assumes homoscedasticity. Bootstrap methods appear to have an advantage over non-bootstrap methods when sample sizes are small. Just how loarge the sample sizes must be.... is unknown....

...A final suggestion is to take advantage of the various plots that were described (earlier in the chapter). They can be invaluable for understanding how groups differ.