3
$\begingroup$

I'm totally not a pro in statistics so i hope the question is not too simple!

I have two groups of patients. One group are smokers, the second are non-smokers.

I want to compare the level of some hormones in the two groups to see if they are influenced by smoking. On many articles I found researchers using Mann-Whitney test to see if the means are significantly different among the two groups. But reading around t-test i'm not sure anymore about which test should I use... unpaired t-test? unpaired t-test with Welch correlation? Mann-Whitney? one-tailed, two-tailed?

Can someone explain me which test to use is this situation and why? (in a way comprehensible by normal people?)

Many Thanks!!!!

Some more info: The two groups are N = 102 (smokers) and 194 (non-smokers), and i red that for N > 30 unpaired t-test and mann-whitney should be similar but applying three tests i got three different p values:

  • Unpaired t-test: p = 0.0245
  • Unpaired t-test with Welch correction: p = 0.0318
  • Mann-Whithney: p = 0.0620

As you can see p values varies a lot and with Mann-Whitney significance is even lost (>0.05)!

Update 2: Also correlation between number of daily cigarettes and hormone level gives strange results: Spearman test gives r = 0.1786 and p = 0.0725, while Pearson gives r = 0.2472 and p = 0.0123. So again with a parametric test the result is significative while with a non-parametric is not!!! I suppose that in this case non-parametric is more correct since we cannot be sure to have the right number of cigarettes, especially since this is a retrospective study.

$\endgroup$
  • 2
    $\begingroup$ You're getting some good information in the answers, I just want to add a comment regarding 1 detail that is peripheral to your main question. You state that you hope to learn if "the level of some hormones... are influenced by smoking". Your setup, as I understand it, does not afford an answer to this question--you can say whether hormone level is associated w/ smoking status, but you cannot infer causality. Inferring causality from observational data is very difficult & you would probably need to work w/ a statistical consultant. $\endgroup$ – gung Apr 2 '12 at 19:38
  • $\begingroup$ you're completely right! :) Again in papers made by doctors you could such a huge amount of statistical inaccuracies that it's amazing how these works can be used for patients management! Every department in my opinion should have a stats professional. You cannot leave this job to students and doctors! $\endgroup$ – Bakaburg Apr 2 '12 at 21:20
4
$\begingroup$

First, a thought: Stop worrying about significance. For example, for the Spearman correlation, you have a (fairly) similar effect estimate to the Pearson test. Same direction, roughly the same magnitude. On a subjective level, you'd say the same thing. The only difference is the p-value. Would your answer, the "takeaway" if the 0.072 slid marginally lower, to 0.05?

When it comes down to it, you'll note your nonparametric results are close to your parametric results. The only reason they feel off is because of a semi-arbitrary line we've put around p=0.05.

Generally speaking however, if you look at your nonparametric tests, they're on the cusp of significance at alpha=0.05. Your problem is that statistical power, while a function of sample size, is not only a function of sample size. It's also a function of the strength of the effect your looking at, and the power of the test itself. Nonparametric tests tend to be somewhat less powered than parametric ones. So what you're seeing is results from a study that has sufficient power for parametric tests, but not for nonparametric ones. Essentially, a study whose sample size (for the strength of effect you're looking at) barely squeaked by.

$\endgroup$
  • $\begingroup$ Thanks! I totally agree with your analysis! The problem is that when you write a paper for publication you have to write if your results are significant or not according to a standard (0.05). I agree that the difference is minimal but i'm at the crossroad between using parametric tests (that i don't think are totally appropriate in this case) and having a formally significant results or use non parametric tests (in my opinion more correct) and have a edge result and have to say that in my opinion is significant enough... With cigarettes correlation hwr, i really cannot say is parametric... $\endgroup$ – Bakaburg Apr 2 '12 at 19:30
  • 2
    $\begingroup$ @Bakaburg Publish in Epidemiology and you don't have to. Problem solved :) $\endgroup$ – Fomite Apr 2 '12 at 19:32
4
$\begingroup$

I'd suggest that you have a look at a graph of the distribution of scores in each of your groups- either a histogram or a density plot, where your score (in this case the hormone level) is on the x-axis and the count or proportion of people with that score is on the y-axis. I'm not sure what software you're using, but the histogram at least should be very easy to produce in most statistics software. Look for asymmetry in the way the scores are spread around the mean, i.e. skew.

The unpaired t-test assumes that both sets of scores are normally distributed with the same variance. The Welch t-test assumes that both sets of scores are normally distributed, but does not assume equal variance. Mann-Whitney doesn't assume either of those things, and is therefore a good choice when your data isn't normally distributed.

By looking at graphs of the distributions of scores, you'll probably get some sense of why the tests are giving different results for your data, because of their different assumptions.

$\endgroup$
  • $\begingroup$ the data is pretty skewed! With sqrt() transformation I can also normalize but nevertheless both groups never pass D'Agostino omnibus normality test. However my biggest problem is that one test pass from not significant to significant if i use t-test instead of mann-whitney. I thought that for high numbers the two tests would give comparable results! $\endgroup$ – Bakaburg Apr 2 '12 at 3:24
2
$\begingroup$

I would suggest that you use a logarithmic transform because hormone concentrations are usually skewed and can never be less than zero. That should make the data at least tolerably normal. You have enough datapoints to test for normality, but if you use a Student's t-test then close enough is good enough because it is quite robust to minor departures from normality. I rarely find the Welch's version of the t-test to be attractive because the differences in variance are either interesting of themselves (e.g. perhaps smoking increases the variability of the hormone levels) or are a consequence of heteroskedaticity that can be corrected with a simple data transformation. In any case you can use a permutations test that makes no distribution assumptions instead of the Student's t-test (it is a little more powerful than the Mann-Whitney test).

See the answers to this question for a lot of information on permutations tests: Significance tests

Judging from your (preliminary) p-values the effect is pretty small. Express the results as a confidence interval based on the log-transformed values with the bounds back-transformed to linear for the reader's convenience.

$\endgroup$
  • $\begingroup$ why logarithmic instead of squared transformation? $\endgroup$ – Bakaburg Apr 2 '12 at 4:38
  • 1
    $\begingroup$ Log is a stronger transformation - it pulls extreme values in towards the mean more than sqrt does. $\endgroup$ – Freya Harrison Apr 2 '12 at 18:47
  • 1
    $\begingroup$ There's nothing the matter with a square-root transformation, Bakaburg. What your comment really concerns is how to choose a transformation to use, if any. Sometimes theory (or experience in an application domain) suggest transformations; logarithms are often implicated in chemical concentrations. You can always analyze the distributions of the two groups. If they are skewed, a transformation can help; choose the one (if any) that simultaneously makes the two distributions approximately symmetric. For more information, research "Box-cox transformation." $\endgroup$ – whuber Apr 2 '12 at 18:53
  • $\begingroup$ i tried with many but no one makes both group normal. We decided to use t-test hoping that the central theorem will make the comparison valid. his mostly because the article is for a medical environment and doctors are not the best in statistic (it would be hard to explain transformations and the rest) But what bugs me more is the test in which significancy get broken going from parametric to non parametric... $\endgroup$ – Bakaburg Apr 2 '12 at 18:59
  • 1
    $\begingroup$ @Bakaburg You should consider using the p value as an index of evidence against the null hypothesis as Fisher intended it to be used rather than comparing it to a binary threshold for significance in the Neyman-Pearson manner. In my opinion, most people are more satisfied by the former even though they are made comfortable by the apparent rigor and certainty of the latter. You might like to read my new paper on the topic ;-) (Brit J Pharmacol DOI: 10.1111/j.1476-5381.2012.01931.x) $\endgroup$ – Michael Lew Apr 3 '12 at 1:14
2
$\begingroup$

If your data conform to the assumptions that a t-test makes (normality, also equal variances if you do not use Welch's version), then a t-test on the data will have greater statistical power than its non-parametric (i.e. less assumption-heavy) equivalent, the Mann-Whitney test. In other words, if your data are normal and have equal variances, and there really is a difference between the smoking and non-smoking groups, then it's perfectly possible to get a significant result from a t-test but not from a Mann-Whitney test. This becomes more likely the smaller the difference between the two groups - even if your sample size is large.

Therefore, if your data are normal or can be made to be normal by transformation, use the t-test. If this is not possible, then use the Mann-Whitney.

However, you should note that while the Mann-Whitney test does not assume normality, it does assume that the distribution of data is similar in the two groups. For example, if the data from smokers are right-skewed and the data from non-smokers are left-skewed, the Mann-Whitney test is not valid. In that case, you'd use randomisation or permutation as Michael Lew says.

$\endgroup$
  • $\begingroup$ Nice to see you back... $\endgroup$ – whuber Apr 2 '12 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.