# How to assess differences between three groups in a pretest - posttest design using SPSS?

I have a dataset that includes one independent variable with 3 levels (group 1, group 2, and group 3) and a repeated dependant variable (pre test score, post test score).

I am using the statistics package SPSS and when I took the method I thought I would use "repeated measures anova" there didn't seem to be a way of splitting the independent variable into the 3 groups. In the data view of the spreadsheet each row represents a participant and so they are grouped by including a numerical value to each group 1 = group 1, 2 = group 2, and 3 = group 3.

I need to compare the mean changes/improvements over the 2 time points (pre, and post) across the groups to assess if there is a significant improvement in any of the groups.

Do you know the most appropriate approach to do this?

The only option I am considering is creating a new spreadsheet where each row represents a participant and each column represents the difference between pre and post trials, and then entering this into a one way independent anova.

• More details on the design would be helpful. 1) What is the nature of the pre & post test, in particular what is there distribution (normal, ordinal, low mean count etc.). 2) Is there an experimental intervention, or are you just interested in the growth for all groups from pre-test to post test? If there is an experimental intervention, what is it's nature (is it dichotomous, you either have the treatment or don't, or is it a continuous treatment)? 3) How many participants (observations) do you have? – Andy W Apr 16 '12 at 11:38
• Groups are treatment groups for a training supplement. Group 1 whey protein, group 2 casein protein, and group 3 placebo (control). the pre and post testing are either side of a 6 week training period. I think the ANOVA style that marius has answered might be right, iv just not leaned how to report the observations of the spas output. If you know of any example reportings of this style that would be amazing. on the output, the "test of within subjects effects" box has an additional line to what i have seen before, and I'm not entirely sure of how i would interpret the results. – Callun Apr 16 '12 at 15:56
• i need to be able to conclude if one or more of the supplementation groups has a greater improvement on the dependant variable. this would most likely be the protein groups as training with protein high diet promotes muscle growth. – Callun Apr 16 '12 at 15:58

This problem seems the classical repeated measures anova problem, assuming ANOVA hyphothesis to hold. Have you tried to have a glance to the repeated measures ANOVA tutorial ("Show me") in the SPSS help?

• is that a feature on the program? I havnt no. Im just having trouble getting my head around it. there is a box you can get up that is labeled "repeated measures" after you click analyse and compare linear models. in this there is an option to add a between subjects factor. but i entered the grouping variable (the column in the data view that groups 1, 2 and 3 for the levels of the independent variable) but I'm not sure this was the correct way to run the statistical test that I'm looking for. – Callun Apr 15 '12 at 19:50

To include both repeated and between-subjects factors in your analysis, you have to run what SPSS calls a repeated measures GLM (don't worry, it's still an ANOVA). Under Analyze, go to General Linear Model -> Repeated Measures. Then you have to define your within-subjects factor (in this case "time", with two levels), and you should be able to enter your between subjects factor in the appropriate box in the next menu.

Your data should be laid out so that each participant corresponds to one row, and you have three columns- Group, identifying which group the participant is in, PreScore and PostScore.

• Thats right, thats exactly where I'm at, however i have not learned fully how to report on the outputs, for the between subjects variable i mean. Is there nothing you think you could sod to help/point me in the right direction? basically need a mini lecture on report writing for this style of anova. If you can help i will be forever grateful! – Callun Apr 16 '12 at 15:50

The analysis of experiments using pre and post test scores has alot of different possibilities for analysis (and such alot of different potential tests). Here I am going to suggest an initial graphical approach to examine your data, and then demonstrate the use of linear regression to assess whether the experimental intervention had a significant impact on "muscle". I'm hoping such a graphical approach is useful regardless of the subsequent ANOVA and/or regression models you subsequently estimate. See this other question, Best practice when analysing pre-post treatment-control designs, for a more general reference on the types of designs possible.

Here I have made up some data, and include code to recreate the fake data as well as make the graphs at the end of the post in SPSS.

To start with, I am going to suggest you simply plot a dot plot of your observations, where the Y axis is the test score (here I have labeled it "Muscle"), and the x axis of the scatterplot represents whether the measure is "pre" or "post" experiment. Here I have also jittered the points (so they don't overlap), and colum paneled the graphic by group.

I suggest you graphically examine your data at the onset because;

• It potentially gives a good initial view of the strength of the hypothesis under consideration. In particular one can generally assess both the central tendency and variance within and between groups. Understanding both are necessary to fit appropriate models later on.
• It may reveal outliers, either on pre-test, post-test (or in the subsequent graphic) from pre to post test.

Other graphics of interest may be lines connecting the pre-test score to post test score (example shown below), or plots summarizing the distributions such as with box plots or points and standard errors.

The above line plot is not as useful to assess correlation (or the general distribution of the test scores) as a scatterplot is, but it is useful to assess outliers in change from pre to post test score.

Further I give some other similar examples (and reference to much discussion) with SPSS code here,Avoid Dynamite Plots! Visualizing dot plots with super-imposed confidence intervals in SPSS and R. I also give examples of making graphs with standard errors and multiple groups in this post on the developerworks SPSS forum. Also to note my blog post I referenced mentions some interesting discussions about what "standard errors" should be plotted in such graphs.

Now, onto making estimates of the experimental effects (and subsequently making inferences about those estimated effects). As I said previously, their are alot of potential types of models you can perform. Such tests are variants of ANOVA (such as MANOVA or ANCOVA), or different types of linear regression models. You also mention conducting an ANOVA on the change scores, which is another consideration.

I won't pretend like I know the best approach (or even have a great understanding of all the potential options). But here I will demonstrate one potential approach, the use of linear regression to estimate treatment effects.

So here we will fit a model of the form,

$Y_{post} = \beta_0 + \beta_1(Y_{pre}) + \beta_2(whey) + \beta_3(casein) + e$

Where - $Y_{post}$ is the post treatment score - $Y_{pre}$ is the pre treatment score - $whey$ is a dummy variable indicating the observation was part of the whey treatment group - $casein$ is a dummy variable indicating the observation was part of the casein treatment group

In the subsequent regression model, you estimate the post treatment score as a linear function of pre treatment score and being in the treatment group whey or the treatment group casein. Because the control group is left as the reference category for the dummy variables, you can interpret the $\beta_2$ and $\beta_3$ estimates as the treatment effects for each group respectively versus the control group (the significance tests with these estimates would also be an example of your "test" you ask for). For help interpreting regression models I would suggest you examine this useful answer on crossvalidated by Gavin Simpson.

I suspect to get much better advice than mine, one would need to have a better understanding of your experimental design in general (as well as regular conventions in your field). A review of the potential modelling possibilities is beyond my ken (and perhaps beyond the reasonable scope of an answer to a question at this site). I hope this is a useful starting point though.

# SPSS code below

****************************************************************************************************************.
*making fake data.
set seed = 10.
input program.
loop #j = 1 to 3.
loop #i = 1 to 30.
compute pre_test = RV.NORM(100,20).
compute group = #j.
end case.
end loop.
end loop.
end file.
end input program.
dataset name sim.

*pre_test has mean 100 and standard deviation 20.

compute dum_group1 = (group = 1).
compute dum_group2 = (group = 2).
compute dum_group3 = (group = 3).
exe.

compute post_test = 70 + 0.3*(pre_test) + 0.2*(dum_group2) + 0.04*(dum_group3) + RV.NORM(0,15).
exe.

compute case_num = \$casenum.

formats post_test pre_test (F3.0).

*now reshaping to long format for graphs.

dataset copy long.
dataset activate long.
varstocases
/make test from pre_test post_test
/index obs_num (test).

*making pre-post scatterplot, column paneling by group.

value labels
group 1 'control'
2 'whey'
3 'casein'.

value labels
obs_num 'pre_test' "pre score"
'post_test' "post score".

* Chart Builder.
GGRAPH
/GRAPHDATASET NAME="graphdataset" VARIABLES=obs_num test group[LEVEL=NOMINAL]
MISSING=LISTWISE REPORTMISSING=NO
/GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
SOURCE: s=userSource(id("graphdataset"))
DATA: obs_num=col(source(s), name("obs_num"), unit.category())
DATA: test=col(source(s), name("test"))
DATA: group=col(source(s), name("group"), unit.category())
GUIDE: axis(dim(1))
GUIDE: axis(dim(2), label("Muscle"))
GUIDE: axis(dim(3), opposite())
SCALE: cat(dim(1), sort.natural(), reverse())
SCALE: linear(dim(2), min(35), max(155))
ELEMENT: point.jitter.normal(position(obs_num*test*group))
END GPL.

*without jitter and lines.
GGRAPH
/GRAPHDATASET NAME="graphdataset" VARIABLES=obs_num test case_num group[LEVEL=NOMINAL] MISSING=LISTWISE
REPORTMISSING=NO
/GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
SOURCE: s=userSource(id("graphdataset"))
DATA: obs_num=col(source(s), name("obs_num"), unit.category())
DATA: test=col(source(s), name("test"))
DATA: group=col(source(s), name("group"), unit.category())
DATA: case_num=col(source(s), name("case_num"), unit.category())
GUIDE: axis(dim(1))
GUIDE: axis(dim(2), label("Muscle"))
GUIDE: axis(dim(3), opposite())
SCALE: cat(dim(1), sort.natural(), reverse())
SCALE: linear(dim(2), min(35), max(155))
ELEMENT: line(position(obs_num*test*group), split(case_num))
END GPL.

*now with standard errors and such - you could super-impose this on jittered scatter - a bit much though (and requires other
manipulation of the data).
GGRAPH
/GRAPHDATASET NAME="graphdataset" VARIABLES=obs_num MEANSE(test, 2)[name="MEAN_test"
LOW="MEAN_test_LOW" HIGH="MEAN_test_HIGH"] group[LEVEL=NOMINAL] MISSING=LISTWISE REPORTMISSING=NO
/GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
SOURCE: s=userSource(id("graphdataset"))
DATA: obs_num=col(source(s), name("obs_num"), unit.category())
DATA: MEAN_test=col(source(s), name("MEAN_test"))
DATA: group=col(source(s), name("group"), unit.category())
DATA: LOW=col(source(s), name("MEAN_test_LOW"))
DATA: HIGH=col(source(s), name("MEAN_test_HIGH"))
GUIDE: axis(dim(1), label("obs_num"))
GUIDE: axis(dim(2), label("Mean test"))
GUIDE: axis(dim(3), label("group"), opposite())
GUIDE: text.footnote(label("Error Bars: +/- 2 SE"))
SCALE: cat(dim(1), include("pre_test", "post_test"), sort.natural(), reverse())
SCALE: linear(dim(2), min(35), max(155))
SCALE: cat(dim(3), include("1.00", "2.00", "3.00"))
ELEMENT: point(position(obs_num*MEAN_test*group))
ELEMENT: line(position(obs_num*MEAN_test*group))
shape.interior(shape.ibeam))
END GPL.

*now onto regression cases.
dataset activate sim.

DATASET ACTIVATE sim.
REGRESSION
/MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT post_test
/METHOD=ENTER pre_test dum_group2 dum_group3.
****************************************************************************************************************.