Comparing smoke and hormones level in two groups of people. Which test?

Question

I'm totally not a pro in statistics so i hope the question is not too simple!

I have two groups of patients. One group are smokers, the second are non-smokers.

I want to compare the level of some hormones in the two groups to see if they are influenced by smoking. On many articles I found researchers using Mann-Whitney test to see if the means are significantly different among the two groups. But reading around t-test i'm not sure anymore about which test should I use... unpaired t-test? unpaired t-test with Welch correlation? Mann-Whitney? one-tailed, two-tailed?

Can someone explain me which test to use is this situation and why? (in a way comprehensible by normal people?)

Many Thanks!!!!

Some more info: The two groups are N = 102 (smokers) and 194 (non-smokers), and i red that for N > 30 unpaired t-test and mann-whitney should be similar but applying three tests i got three different p values:

• Unpaired t-test: p = 0.0245
• Unpaired t-test with Welch correction: p = 0.0318
• Mann-Whithney: p = 0.0620

As you can see p values varies a lot and with Mann-Whitney significance is even lost (>0.05)!

Update 2: Also correlation between number of daily cigarettes and hormone level gives strange results: Spearman test gives r = 0.1786 and p = 0.0725, while Pearson gives r = 0.2472 and p = 0.0123. So again with a parametric test the result is significative while with a non-parametric is not!!! I suppose that in this case non-parametric is more correct since we cannot be sure to have the right number of cigarettes, especially since this is a retrospective study.

Answer 1

First, a thought: Stop worrying about significance. For example, for the Spearman correlation, you have a (fairly) similar effect estimate to the Pearson test. Same direction, roughly the same magnitude. On a subjective level, you'd say the same thing. The only difference is the p-value. Would your answer, the "takeaway" if the 0.072 slid marginally lower, to 0.05?

When it comes down to it, you'll note your nonparametric results are close to your parametric results. The only reason they feel off is because of a semi-arbitrary line we've put around p=0.05.

Generally speaking however, if you look at your nonparametric tests, they're on the cusp of significance at alpha=0.05. Your problem is that statistical power, while a function of sample size, is not only a function of sample size. It's also a function of the strength of the effect your looking at, and the power of the test itself. Nonparametric tests tend to be somewhat less powered than parametric ones. So what you're seeing is results from a study that has sufficient power for parametric tests, but not for nonparametric ones. Essentially, a study whose sample size (for the strength of effect you're looking at) barely squeaked by.

Answer 2

I'd suggest that you have a look at a graph of the distribution of scores in each of your groups- either a histogram or a density plot, where your score (in this case the hormone level) is on the x-axis and the count or proportion of people with that score is on the y-axis. I'm not sure what software you're using, but the histogram at least should be very easy to produce in most statistics software. Look for asymmetry in the way the scores are spread around the mean, i.e. skew.

The unpaired t-test assumes that both sets of scores are normally distributed with the same variance. The Welch t-test assumes that both sets of scores are normally distributed, but does not assume equal variance. Mann-Whitney doesn't assume either of those things, and is therefore a good choice when your data isn't normally distributed.

By looking at graphs of the distributions of scores, you'll probably get some sense of why the tests are giving different results for your data, because of their different assumptions.

Answer 3

If your data conform to the assumptions that a t-test makes (normality, also equal variances if you do not use Welch's version), then a t-test on the data will have greater statistical power than its non-parametric (i.e. less assumption-heavy) equivalent, the Mann-Whitney test. In other words, if your data are normal and have equal variances, and there really is a difference between the smoking and non-smoking groups, then it's perfectly possible to get a significant result from a t-test but not from a Mann-Whitney test. This becomes more likely the smaller the difference between the two groups - even if your sample size is large.

Therefore, if your data are normal or can be made to be normal by transformation, use the t-test. If this is not possible, then use the Mann-Whitney.

However, you should note that while the Mann-Whitney test does not assume normality, it does assume that the distribution of data is similar in the two groups. For example, if the data from smokers are right-skewed and the data from non-smokers are left-skewed, the Mann-Whitney test is not valid. In that case, you'd use randomisation or permutation as Michael Lew says.

Answer 4

I would suggest that you use a logarithmic transform because hormone concentrations are usually skewed and can never be less than zero. That should make the data at least tolerably normal. You have enough datapoints to test for normality, but if you use a Student's t-test then close enough is good enough because it is quite robust to minor departures from normality. I rarely find the Welch's version of the t-test to be attractive because the differences in variance are either interesting of themselves (e.g. perhaps smoking increases the variability of the hormone levels) or are a consequence of heteroskedaticity that can be corrected with a simple data transformation. In any case you can use a permutations test that makes no distribution assumptions instead of the Student's t-test (it is a little more powerful than the Mann-Whitney test).

See the answers to this question for a lot of information on permutations tests: Significance tests

Judging from your (preliminary) p-values the effect is pretty small. Express the results as a confidence interval based on the log-transformed values with the bounds back-transformed to linear for the reader's convenience.