Dear statistics experts,

I have trouble to find a sensible statistical approach to back up some very obvious (at least to my eyes) interpretation of a dataset (see descriptive plot below).

I measure a response to two different stimulus (Stimulus 1 and 2) at 3 different intensities (weak medium high), in 2 different populations (population 1 and 2).

What I see and would like to support with a statistical test, is that overall the response exhibit a dynamic range along the intensity range, but more importantly, that this range is very steep for stimulus 1 in the population 2 but, not in population 1.

In terms of design, I have

  • a continuous dependent variable (my response).
  • two categorical variables with two levels each (stimulus and population)
  • one ordinal variable with 3 levels (stimulus strength)

Samples are not independent, as response to each stimulus/strength pair is assessed on the same subjects in the two populations.

I thought of using a mixed effect ANOVA with subject as a random variable to account for the repeated measure.

First problem : How to measure the departure from the homoscedasticity assumption ? A Levene's test reject the null hypothesis of equality of variance between groups. Anova is said to be robust to violations of its assomptions, but is there a way of judging the severity of this violation, especially given my unbalanced design?

Second, would my biological statements be reinforced if I was to find a significant 3-way interaction population x stimulus x strength ? I have two concerns :

  • Does a 3-way interaction p-value have any meaning on such a design and especially given my first point (disparity in variance) ?

  • even if this effect was real, is it allowed (not to say feasible), to conclude anything biologically relevant from a three-way significant interaction, which essentially mean, (correct me if I am wrong), that at least one of the two-way interaction between a pair of factors is influenced by the third one ? Specifically here, I would like to test if the stimulus x strength interaction is influenced by the population factor. Should I then test for significance of this two-way interaction beforehand ?

My meditations have carried me far beyond what my very basic understanding of statistics allow me to comprehend.

Thanks in advance for your input,


enter image description here


1 Answer 1

  1. The F-test is fairly robust to heterogeneity of variance (HV) if all sample sizes are equal (or similar). In any case you should not rely on Levene's test for testing HV because the test is not powerful (i.e., tends to produce more type II errors). Check the variance ratio: $Max(s^2)/Min(s^2)$. If the ratio is less than 4, HV should be assumed (Maxwell and Delaney, 2004); otherwise, you may want to transform the data to see if the results are consistent.

  2. A significant 3-way interaction means, as you correctly pointed out, the 2-way interactions is influenced by the other factor. In fact, the plot you provided suggests that the stimulus x strength interaction is different across populations. Specifically, the evidence of the simple interaction between stimulus x strength seems strong within population 2 (i.e., DV goes up as strength goes up with stimulus 1 but not with stimulus 2) but seems very week within population 1 (i.e., lines are almost parallel across stimulus conditions).


To explore the locus of the 3-way interaction, I would use a linear contrast for stimulus intensity: small (?), moderate, strong = -1, 0, 1. Then you can express the simple interaction within each population as $Linear Change_{Stimulus1} - Linear Change_{Stimulus2}$. This value indicates how linear changes are moderated by the stimulus type with positive values indicating greater linear changes in stimulus 1 than stimulus 2. From the plot it appears that this value would be a small negative value in population 1 but a large positive value in population 2, hence the 3-way interaction.

  • $\begingroup$ 1. Thank you Masato, I checked and my highest variance ratio accross all levels is 2.7 $\endgroup$
    – Benjamin
    Commented Sep 4, 2014 at 19:50
  • $\begingroup$ 2. I performed a multifactorial anova and found a significant effect of population x stimulus x strength. However, when I went on to investigate the conditional simple interactions (stimulus x strengh in each population), I also found a significant stimulus x strength interaction in population 1 (which is actually visible on the plot). Is there a way to quantity these two effects, to allow direct comparison ? As you pointed out, the interaction is very weak in population 1 and very strong in population 2. $\endgroup$
    – Benjamin
    Commented Sep 4, 2014 at 20:06
  • $\begingroup$ 3. What you propose sounds great, but I have no idea how to implement it on the data : How do I calculate the linear change for one stimulus ? Is it something like the slope of the linear regression of the DV on my "linearly contrasted" stimulus intensity ? $\endgroup$
    – Benjamin
    Commented Sep 5, 2014 at 7:36
  • $\begingroup$ That's basically it. You can use the coding -1, 0, 1: week, moderate, strong. $\endgroup$ Commented Sep 5, 2014 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.