I would perform a regression if I were you. With regressions you not only get a significance test, you also get an estimate of the differences between groups. You don't get this with an ANOVA. You can code the treatments as several dummy-coded variables. If you have four treatments, control
, treatment1
, treatment2
, treatment3
, then you need three dummy variables. These are binary variables coded 0 or 1 depending on whether the observation in question belongs to the dummy group. It's easiest to explain if you can see the data. Here's some fake data. See how the treatment1_dummy
variable is coded 1 if the observation falls in the same row as treatment1
in the group
variable and 0 for all the other observations, treatment2_dummy
is coded 1 if the observation belongs falls in the same row as treatment2
in the group
variable and so on.
outcome group treatment1_dummy treatment2_dummy treatment3_dummy
1 -0.17792783 control 0 0 0
2 1.36817934 control 0 0 0
3 -0.04472688 control 0 0 0
4 -1.07506861 control 0 0 0
5 -0.18948026 control 0 0 0
6 4.73309782 treatment1 1 0 0
7 4.23008616 treatment1 1 0 0
8 3.98634961 treatment1 1 0 0
9 5.07688582 treatment1 1 0 0
10 6.04681029 treatment1 1 0 0
11 9.89827005 treatment2 0 1 0
12 10.01896885 treatment2 0 1 0
13 10.63345666 treatment2 0 1 0
14 9.86221136 treatment2 0 1 0
15 10.23407764 treatment2 0 1 0
16 13.52274846 treatment3 0 0 1
17 17.07765091 treatment3 0 0 1
18 15.17727922 treatment3 0 0 1
19 15.17477509 treatment3 0 0 1
20 16.60549137 treatment3 0 0 1
I created the data so the control
group's mean outcome value is around 0, the treatment1
group's mean is about 5, treatment2
group's is around 10, and treatment3
is around 15. The means would look like this
Now you just do a simple linear regression, entering the dummy variables in as predictors. I'm doing this in R but this method will be possible in all software packages.
mod <- lm(outcome ~ treatment1_dummy + treatment2_dummy + treatment3_dummy, data = df)
summary(mod)
# Output
# Coefficients:
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) 0.06526 0.44090 0.148 0.884
# treatment1_dummy 4.55927 0.62353 7.312 1.74e-06 ***
# treatment2_dummy 10.73874 0.62353 17.223 9.46e-12 ***
# treatment3_dummy 14.57315 0.62353 23.372 8.55e-14 ***
These are uncorrected for multiple comparisons. Because there is no dummy variable for the control
group, the coefficients in the output table above represent the estimated difference in average score between the treatment group in question and the reference group (i.e. the control group). The '(Intercept)' coefficient tells you the estimated score in the control group (i.e. unlike the other coefficients this is just the estimated average score in that group, not a test of between-group difference in average score).
Bonferroni is fine for error correction but, like ANOVA, a little old-fashioned. It can be overly conservative and, when there are lots of comparisons, end up correcting out legitimate effects! A really good method is the Benjamini-Hochberg procedure (also known as the false discovery rate, or fdr). Not sure whether SPSS has it as a default but would be surprised if there were no SPSS macros out there that can help you do it.
Here is the output from all pairwise group comparisons, controlled using the false discovery rate method
Linear Hypotheses:
Estimate Std. Error t value Pr(>|t|)
treatment1 - control == 0 4.5593 0.6235 7.312 2.09e-06 ***
treatment2 - control == 0 10.7387 0.6235 17.223 2.84e-11 ***
treatment3 - control == 0 14.5731 0.6235 23.372 5.13e-13 ***
treatment2 - treatment1 == 0 6.1795 0.6235 9.911 4.67e-08 ***
treatment3 - treatment1 == 0 10.0139 0.6235 16.060 5.47e-11 ***
treatment3 - treatment2 == 0 3.8344 0.6235 6.150 1.40e-05 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Adjusted p values reported -- fdr method)
Notice the first three comparisons are the same as in the first output. The error correction hasn't changed the p-values much because they were so significant already. But this is a neat way to get what you want, obtain significance tests as well as estimates of unstandardised effect size, and control for multiple comparisons.