0
$\begingroup$

I would like to get a proper justification for choosing one of the three of the various methods used to determine the causes of interactions in post-hoc procedures for multivariate ANOVA. I have chosen R and SPSS as examples, but it seems to me that this is irrelevant.

Consider a simple example of a 3x2 ANOVA, three drugs (drug A, drug B, and placebo) and two gender groups (M/F). We obtain a significant interaction effect, we can proceed to the post-hoc procedure.

1)

First, we can consider comparing "everything with everything" - we have (in some way) 6 sub-groups and we simply compare each with each other, creating a matrix something like this (values ​​are sample p-values):

    drug A M    drug B M    placebo M   drug A F    drug B F    placebo F
drug A M                        
drug B M    0,321                   
placebo M   0,251   0,251               
drug A F    0,181   0,181   0,181           
drug B F    0,111   0,111   0,111   0,111       
placebo F   0,041   0,041   0,041   0,041   0,041   

This is the default approach in Statistica software (for example), or some analysis in R.

2)

We can determine differences within one grouping variable between groups of another variable:

drug A M    vs drug A F     0,321
drug B M    vs drug B F     0,251
placebo M   vs placebo F        0,181

and

drug A F    vs  drug B F    0,321
drug B F    vs  placebo F   0,251
placebo F   vs  drug A F    0,181
drug A M    vs  drug B M    0,111
drug B M    vs  placebo M   0,041
placebo M   vs  drug A M    0,251

This is the default approach in SPSS software.

3)

We can simply accept single comparisons consistent with our theory, combine groups, etc.

drug A M + placebo F   vs drug A F      0,321
drug B M    vs drug B F + placebo M     0,251

I remember from statistics courses that the first approach had no name, the second was called a "simple effect(s)", and different variants of the third were called "contrasts". If these names are incorrect - then of course I will gladly accept any knowledge.

These three approaches, seem to me to, give different results, more precisely different levels of statistical significance using Bonferroni methods. This is not surprising, if the first approach consider 15 compared subgroups, in the second 3 or 6, in the third 2. This gives, according to the rules of Bonferroni correction (multiplication of the p-result by the number of comparisons), completely different levels of significance, with the first one simply being the most conservative. I tested it in R, SPSS and Statistica and calculated what I could by hand.

Due to this, I have a series of questions, primarily about the names of these techniques.

  1. Can any of these approaches be considered generally correct or incorrect? If it depends on the type and interpretation of the data, then which? If it depends on the researcher's theory, then how is it proven to be roughly correct?
  2. Since these three approaches yield different levels of significance, how can one justify choosing one over the other? Many studies report and interpret comparisons from Example 2 because they seem to answer the question about the nature of the interaction effect, but is this the right way to go?

I don't expect all the answers, any advice will be valuable to me.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.