6
$\begingroup$

I have an experiment producing results (dependent variables) that don't pass tests of normality, thus I am testing hypotheses using non-parametric tests. My DVs are continuous, while my factors (independent variables) are ordinal or nominal. I've been using the Kruskal-Wallis test and Friedman test (using Matlab). Most of the time I am only interested I testing 2 IVs for significant effects, though sometimes I test 3.

I would like to know whether there are any significant interaction effects on the DV between my IVs. Normally I'd use a 2-way ANOVA to do this, however that's not appropriate given the non-normal distributions. I don't wish to use transformation of my IVs, nor go ahead with ANOVA despite non-normality.

How can I find which interaction effects are significant?

What non-parametric test could I use?

Hope someone can help.

Nick

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ In what way is the data "non-normal"? skewed? "fat tails"? outliers? You may be able correct this by using a different distribution (e.g. t or cauchy for fat tails and outliers, and rescaled beta distribution for skewness, off the top of my head), and save some power in your test. $\endgroup$
    – probabilityislogic
    Feb 17, 2011 at 14:09
  • 1
    $\begingroup$ The data is not normal in various ways. Some are skewed (Poisson type distributions), some are like the combination multiple normal distributions, several just have a few extreme outliers. The issue is that I have been asked to specifically test for normality (I use the Chi-squared goodness of fit test). The data strongly reject the null hypothesis of normal distribution. Transformation is not an option - and in many cases doesn't work anyway. I agree this should be something to try, but I have been disallowed this option, not my choice. Thanks though! $\endgroup$
    – Nick
    Feb 19, 2011 at 1:25
  • 1
    $\begingroup$ Are any of your DVs actually Poisson-distributed? I.e., are they counts of the frequencies of events over successive periods of time, and does the mean roughly equal the variance? If so, you may be able to use Poisson regression to test the interactions you're interested in. $\endgroup$
    – rolando2
    Feb 25, 2011 at 3:23
  • $\begingroup$ I will look into that. Any tips on where to read about Poisson regression - with examples? I'd nevertheless still like to hear about any more general non-parametric interaction tests. $\endgroup$
    – Nick
    Feb 27, 2011 at 3:59
  • $\begingroup$ What about Tukey's median polish? $\endgroup$
    – aL3xa
    Mar 26, 2011 at 20:13

2 Answers 2

Reset to default
2
$\begingroup$

Non-parametric tests are likely to be less powerful than parametric tests and thus require a larger sample size. This is annoying because if you had a large sample size, sample means would be approximately normally distributed by the central limit theorem, and you thus wouldn't need non-parametric tests.

Look at generalized linear models, of which least squares and Poisson are special cases. I've never found a text that explains this particularly well; try talking to someone about it.

Look at non-parametric methods if you feel like it, but I have a hunch that they won't help you much in this case unless you're using ordinal data or a large set of very bizarrely distributed data.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ McCullagh and Nelder is pretty good, although somewhat mathematically dense. I figured out how to use GLM's from the first 100 pages or so. Also, Gelman and Hill discuss GLM's in their Applied Regression Models and this book is pretty good for taking you from the very basics right through to Multilevel models using lme4 and Bugs. $\endgroup$
    – richiemorrisroe
    Mar 24, 2011 at 13:40
  • $\begingroup$ @richie +1 OMG It is! I got it last week! I love it! $\endgroup$
    – Avery Richardson
    Apr 15, 2011 at 20:18
1
$\begingroup$

I had the same questions and made some research. I came across some texts that seem to offer solutions but I ahve to admit that I did not seriously apply them until now.

  1. Feller, A., Holmes, C.C., 2009. Beyond toplines: Heterogeneous treatment effects in random-ized experiments.
  2. Leys, C., Schumann, S., 2010. A nonparametric method to analyze interactions: The adjusted rank transform test. Journal of Experimental Social Psychology 46 (4), 684–688.
  3. Sawilowsky, S.S., 1990. Nonparametric tests of interaction in experimental design. Review of Educational Research 60 (1), 91–126.

At least for 1. it seems that you rely on a sufficiently large sample size. They analyze a dataset which has between 38,00 and 190,000 observations per treatment. If you are working with experimental data from a laboratory and are from the behavioral field, this is probably not very helpful. However, I find their analysis of interaction effects, especially their graphical interpretation, very vivid and intuitive.

The 2. text discusses one of the approaches that are discussed in 3. It has been a while since I read the latter paper, but if I remember correctly, the author presents some approaches to analyze interactions non-parametrically in a practical way. As probabilityislogic said, people often criticize that non-parametric tests of interactions lack power. However, Sawilowsky (1990) states that "The review shows that these new techniques are robust, powerful, versatile, and easy to compute." On the other hand, the text is quite old ;)

Other approaches, of which I only know the name, are Finite Mixture Models and Latent Class Regression Models. One of the two is a special form of the other, but I do not remember which one.

Hope this helps.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.