# pretest-posttest experimental design: which anaysis tests to use?

I am new to this website, and I hope to find help. I am writing my MSc thesis on phishing education.

I have three groups and use pretest-posttest experimental design

1. Control
2. Training 1
3. Training 2

It aims to test whether anti-phishing training is effective in educating users to recognise phishing emails. I also use few control variables such as demographics ones, IT self-efficacy, etc.

The dependent measure is the final score.

These are the tests that I used:

• One-way Anova followed by Post hoc Tukey and paired t-test to compare the scores between pre and post-test. However, it turned out that my post-test is more difficult than my pre-test; this is a big limitation (I don't know how to address this issue)
• Signal detection theory: One-way Anova followed by Post hoc Tukey and paired t-test was used to examine: False negatives and false positives rates/Sensitivity and criterion.

I would like to know whether my tests are sufficient or I need additional analysis?

• What’s the outcome? Score of what? Aug 12, 2021 at 3:51

$$y_{post} = \beta_0 + \beta_1y_{pre} + \beta_2 x_{T_1} + \beta_3 x_{T_2}$$
If you want to know which intervention is "best" you can use a Tukey's HSD but honestly the type one error rate is not inflated all that much. You could probably just use a smaller $$\alpha$$ in your initial model and end up with a 5% type one error rate after the multiple comparisons.