3
$\begingroup$

I have an experiment where I'm testing the effect of a drug on behavior, measured in seconds. My drug group doesn't have normal distribution, probably because of a bimodal pattern: 6 of the 8 are clustered (between 100 and 150 s), while the other 2 are much higher (275 and 300 s). We believe this is because the drug worked for those 6, and not the others.

Furthermore, a One-Way Anova between the 3 groups failed Levene's test for equal variances (I also ran Welch's t-test, which was significantly diff.) However, an independent T-test between the 2 groups that I'm looking at shows no sig. diff. in Levene's Test, and the One-way Anova says there is a significant decrease in time between this group and a control group using Tukey's, which is what I'm looking for (the 3rd group is another control, but I'm not really interested in that one).

All 3 have an n= 8.

Where do I go from here?

Edit:

My design involves 24 animals split into 3 drug groups, 8 each: a control drug, saline, and the drug I am testing. I am seeing if there's a significant decrease in time between the drug group I'm testing and the saline group.

$\endgroup$
  • $\begingroup$ It is really tough to give you good advice based on such scant characterization. What are the three groups? And, BTW, you might try transforming the data. For example, time to effect may be skewed because of how events propagate in the body, i.e., non-linearly. Try taking the reciprocal of time, and the logarithm of time, one of those will be more normal. However, the data you have to too scant for good characterization of normality, so, you would be better off using non-parametric methods. For example, Conover rather than Levene. Wilcoxon rather than t-test, etc. $\endgroup$ – Carl Oct 28 '16 at 19:49
  • $\begingroup$ @Carl Transforming the data to normalize the variable for one group is likely to de-normalize the variable for other groups. $\endgroup$ – Alexis Oct 28 '16 at 20:01
  • $\begingroup$ What are the three groups? I cannot help what you do not explain. $\endgroup$ – Carl Oct 28 '16 at 20:05
  • $\begingroup$ Sorry, so a little more background: im using stressed rodents here. One is a group given a drug that is known to nullify the effects of stress, one is given saline, and the other is given the drug I'm testing. I will try to transform my data in a bit, and also look at those methods you mentioned. Thank you $\endgroup$ – Ali D Oct 28 '16 at 20:05
  • 1
    $\begingroup$ Ali: the Kruskal-Wallis test makes no assumptions about your distributions other than that measures within each variable are i.i.d. Technically the nonparametric tests assume continuous data, but there are adjustments for ties when using them for discrete data. So have at! $\endgroup$ – Alexis Oct 29 '16 at 19:17
4
$\begingroup$

If the assumption of normality for one-way ANOVA does not hold, you can turn to a nonparametric analog to the one-way ANOVA: the Kruskal-Wallis test. Just as the assumption of normality underlying the unpaired t test may not be met, thus motivating the use of the rank sum test, onne can then use Dunn's test, or the more powerful (but less well known) Conover-Iman test to conduct post hoc pairwise tests if one rejects the omnibus Kruskal-Wallis test's null hypothesis.

In their most general form the nonparametric tests (Kruskal Wallis, rank sum, Dunn's, etc.) do not assume equal variances among groups. Instead, they test:

$$H_{0}:P(X_{A}>X_{B})=0.5$$

with

$$H_{a}:P(X_{A}>X_{B})\ne0.5$$

Or in words: the null hypothesis is that the probability that a randomly selected observation from group A is greater than a randomly selected observation from group B equals one half. The alternative is that the probability is not one half. For the Kruskal-Wallis test, the null hypothesis is that the probability that a randomly selected value from any group is greater than a randomly selected observation from any other group equals one half, with the alternative that at least one group that has a probability not equal to one half for being greater than a randomly selected value from another group.

One can interpret these as tests of location shift, median difference, or mean difference if the variances for all groups are all equal and the shapes of the distribution are the same (this is a pretty stringent requirement!), but nonparametric tests do not require such assumptions to use.

I have published a software package to perform Dunn's test for R (dunn.test), and Dunn's test for Stata (dunntest), and a software package to perform the Conover-Iman test for R (conover.test), and the Conover-Iman test for Stata (conovertest). Both packages correct for ties, and implement an array of familywise error rate and false discovery rate adjustments for multiple comparisons.

References
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.

Conover, W. J. (1999). Practical Nonparametric Statistics. Wiley, Hoboken, NJ, 3rd edition.

Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.

$\endgroup$
  • $\begingroup$ I ran the Kruskal-Wallis test, and got an Asymp. sig.= .000. How do I go about doing Dunn's or Conover's test on Spss? $\endgroup$ – Ali D Oct 28 '16 at 20:53
  • $\begingroup$ Alexis, doesn't the K-W test assume that variances are equal, which is not true in my case? $\endgroup$ – Ali D Oct 29 '16 at 5:18
  • $\begingroup$ @AliD No, the no parametric tests (Kruskal Wallis, rank sum, Dunn's, etc.) do not assume equal variances. They test $H_0: P(X_A > X_B) = 0.5$ with $H_a: P(X_A > X_B) \ne 0.5$. Or in words: the null hypothesis is that the probability that a randomly selected observation from group A is greater than a randomly selected observation from group B equals one half. The alternative is that the probability is not 0.5. One can interpret these as tests of location shift, median difference or mean difference if the variances are all equal, but the test does not require that to use. $\endgroup$ – Alexis Oct 29 '16 at 19:07
  • $\begingroup$ @AliD So you can use Kruskal-Wallis plus Dunn's or Conover-Iman. As to how: I have published software for both Dunn's test and the Conover-Iman test for R and for Stata. Search for dunn.test or conover.test (R) or dunntest or conovertest (Stata). $\endgroup$ – Alexis Oct 29 '16 at 19:10
  • $\begingroup$ Great, thank you for the thorough explanation. As far as a follow-up test goes, you mention the rank sum test; do you mean the Mann-Whitney test by this? If so, how does that compare to the other 2 that you mentioned as far as my results goes? I don't believe either is available on SPSS, but MW is so I used that (really cool that you published those though!). $\endgroup$ – Ali D Oct 31 '16 at 17:10
1
$\begingroup$

TRANSFORM This was already mentioned in one of the comments to the question, but let me emphasize it. Transforming often makes sense. And it isn't just a mathematical trick -- it is just another way to look at the data.

You measured duration in seconds. The reciprocal of duration is proportional to speed or velocity. That may be a more natural way to think about this effect. Compare speeds instead of comparing durations.

CONTROLS You don't give the details, but it sounds like one of the controls is just a methods control. If the purpose of that control is just to make sure the method worked as expected (that the control injection of saline didn't by itself do much to the outome you measure), then there is no reason to include those results in the final analysis. It is ok to check controls to make sure the experimental method "worked", then ignore those controls when actually analyzing the data on the experimental effect.

$\endgroup$
  • $\begingroup$ Harvey, if two groups are normal, and the third group is not normal then transforming the third group to normal will either (1) denormalize the first two groups, or (2) mean you are conducting ANOVA on two variables measures in one unit, and a third variable measured in a different unit. This is a clear-cut situation appropriate to nonparametric methods. $\endgroup$ – Alexis Nov 1 '16 at 21:21
  • $\begingroup$ @Alexis Of course you have to transform all the data the same way. Whether that would be helpful in this case is a question that can't be answered without knowing more about the scientific context and also seeing the data. $\endgroup$ – Harvey Motulsky Nov 9 '16 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.