What methods to use in pre and post testing?

Question

I am currently running a pre- and post-intervention study. My hypotheses are if the intervention has an effect and if personality traits moderate the effect of the intervention.

I have an initial sample of $n=253$ participants and a follow up sample including only $n = 102$ participants. My analysis plan is to employ t-tests, to compare effect, then hierarchical regression, and finally a moderation analysis.

My questions are:

• Do I discuss just the 102 participants within my analyses?

• Do I use the post test score as the dependent variable?

• Do I need to use the pre-test score as a covariate in the hierarchical regression?

• Do I need to use the pre-test score as a independent variable or compute a difference score between pre and post?

If someone could point me towards a paper using similar analyses that would be great as I have been at this for weeks.

Answer 1

I will assume that the same subjects took the pre and post tests? So you have paired data. Your plan was to use a t-test, so you should use a paired t-test (a good thing, it is pretty powerful). Therefore you can only report on the 102 participants who completed the 2 tests (and 102 is a very reasonable sample size for a paired t-test). Therefore the post test score is not the dependent variable, because you will be looking at the difference in scores (your only ! variable). Therefore the pre-test score is not a covariate (again, you will just look at the difference of scores).
Once you start looking at how personality traits may affect the scores, things can get more complex; you could just look at how some personality traits associate with low/high pre-test scores, or low/high post test scores, or just low/high score differences. This depends in part on what the data says, but also on what may be of interest in your specific domain. However, be careful to use multiple comparison corrections as appropriate, if you start using several exploratory tests.

Answer 2

I will address your questions individually:

Do I discuss just the 102 participants within my analyses?

I would advise against just using these participants. Your analysis will be obviously biased because of survivorship. A solution for your case would be to use a principled form of imputation, like multiple imputation, full information maximum likelihood, or random forest imputation. Justification can be found in this article.

Do I use the post test score as the dependent variable?

Typically in a regression that requires an estimation of the change in scores, you would model the score as the response, with a dummy variable to toggle on/off the timing of the score (pre/post). This allows you to see what the conditional mean of the response is for each time. This also allows you to include other covariates (rather than treat this as an isolated model or t-test).

Do I need to use the pre-test score as a covariate in the hierarchical regression? Do I need to use the pre-test score as a independent variable or compute a difference score between pre and post?

See the above comment. The only covariates to include would be the other variables you mentioned (e.g. traits, intervention). Do not use a difference score. With these questions out of the way, you noted the following:

I have a 253 pre sample and follow up is only 102. My analysis plan is t-tests, to compare effect, then hierarchical and finally moderation analysis.

You seem to have three different methods you are employing here. I would make sure that you have very specific hypotheses that only these different methods can address. If your entire research question can be effectively described by a single model (e.g. regression), then I would instead go with that.