Is it possible to compare means of pre & post intervention questionnaire for only 3 participants?

Question

I am completing my dissertation which was to determine if a 6-week expressive writing intervention would reduce aggressive behaviors & increase emotion regulation in students. My first variable (aggression) is being graphically analyzed. My second variable (emotion regulation) is being assessed by comparing pre-intervention scores on the Emotion Regulation Questionnaire to post-intervention scores. This is a single subject design so I only have 3 participants. What test do I run to compare means of pre/post test with three separate individuals? (Not groups).

Answer 1

In the absence of prior information on what you would expect without intervention, comparing pre- and post-intervention values is not a suitable approach for assessing the causal effect of an intervention. Of course, if you can from other studies in the literature construct, say, a meta-analytic predictive distribution for what you'd expect without intervention, or if you could elicit a prior distribution from experts, then you could potentially do something meaningful even without a control group within your study.

Other than that, a low number of participants is not in principle an issue, if we were to expect an enormously large effect relative to the natural variability of the outcome measure. There's a number of potential problems though:

• Your power to, say, have a statistically significant effect will potentially be small, unless you expect the intervention effect to be enormous.
• If the power of your experiment is very small and a priori it's - say - 50-50 whether your intervention does much, then any "statistically significant" or otherwise notable findings are much more likely to be false positives with massively overestimated effect sizes.
• You could run into issues with some traditional frequentist analysis methods (e.g. collinearity, a residual standard error of 0 etc. are not so unlikely with N=3). You might be better of taking a Bayesian approach (also, of course, when you try to incorporate what you might know about what would happen without intervention).