1
$\begingroup$

I measured a variable during pre- and post-sessions and my research question is whether the change from pre to post is different across various conditions (e.g. (c=1,a=1), (c=2,a=1), (c=1,a=2), (c=2,a=2) etc) and if so, across which.

How can I test whether the differences between pre- and post are significantly different across conditions?

This may sound easy, however, the pre- and post-session measurements are not paired in any meaningful way. Hence, using an arbitrary pairing to compute the differences and then performing an ANOVA for the differences across conditions does not make full use of the dataset.

Alternatively, I think a multi-way ANOVA could be applied with factors (pre/post), c and a (for the above example). In that case, however, it is unclear to me how to figure out across which of the conditions the change from pre to post is significantly different. Posthoc pairwise comparisons would not directly provide such information.

Thank you very much in advance for any help!

$\endgroup$
6
  • $\begingroup$ I'm really unclear on your setup. Most critically, if you have a single variable measured pre- and post-treatment, how is it that your data are not paired in a manner appropriate for a paired t-test? $\endgroup$
    – Upper_Case
    Jul 25, 2017 at 13:44
  • $\begingroup$ Thanks for the reply. The variable is measured during 20 trials pre and 20 trials post but there is no relation between note first trial pre and the first trial post etc. Hope its clearer now. $\endgroup$
    – carsten
    Jul 25, 2017 at 14:41
  • $\begingroup$ Your data seems like it must be at least theoretically paired, but that the pre- and post-treatment measurements that correspond to a particular subject are unmatched. Is that right? $\endgroup$
    – Upper_Case
    Jul 25, 2017 at 20:28
  • $\begingroup$ For each subject there are 20 measurements pre (within a short time) and 20 measurements post (again within a short time). The question is whether the difference pre-post varies significantly across different factors. There are two factors: a and c; both of which are binary. For each subject there are always measurements for a=0 and a=1 but not necessarily for c=0 AND c=1. For some subjects only c=0 and for others only c=1 was measured. In addition there are relatively few subjects and therefore I would like to make use of the 20 pre and 20 post measurements. Thanks for the help! $\endgroup$
    – carsten
    Jul 25, 2017 at 22:32
  • $\begingroup$ And are these 20 measurements per subject time-independent (so that it might make sense to average them and you are taking 20 measurements to measure one "thing"), or time-dependent? $\endgroup$
    – Upper_Case
    Jul 26, 2017 at 15:15

2 Answers 2

0
$\begingroup$

Based on your statement, I think paired t test can be done to assess the pre and post session measurements as the values belong to the same sample.

$\endgroup$
4
  • $\begingroup$ Thanks for the response but I am interested in testing whether post minus pre changes significantly across conditions. $\endgroup$
    – carsten
    Jul 25, 2017 at 14:40
  • $\begingroup$ If you want to check whether there is significant difference between the Pre treatment values and Post treatment values then paired t test is the solution. $\endgroup$
    – Sam Gladio
    Jul 25, 2017 at 14:46
  • $\begingroup$ Yes but I want to check whether the difference between pre and post varies with conditions in a situation where there is no meaningful pairing between the pre and post samples. $\endgroup$
    – carsten
    Jul 25, 2017 at 15:50
  • $\begingroup$ Do you mean to say that pre and post samples are independent? i.e. A in pre sample is independent of A in post sample. If yes, then use manova on 2 inputs(pre and post, A and B and C...) and one output. $\endgroup$
    – Sam Gladio
    Jul 25, 2017 at 20:23
0
$\begingroup$

Someone more experienced than me can probably weigh in more usefully, but to me this sounds like a difficult data set to work with. That your data are unpaired, despite a research question that seems based in within-subject differences, could do a lot to mask the actual effect of each treatment. I agree that arbitrarily pairing data is not desirable in this case.

Assuming that I understand the data you have correctly, I think that your best bet would be to do unpaired, two-tailed t-tests on the pre- and post-treatment measurements of your variable for each condition. This covers a "did the treatments have any effect?"-style question in a fairly typical way.

Comparing between conditions seems a lot iffier to me given what you have described. I'm not clear on how subjects were assigned to different conditions, but when only some subjects have measures for condition c, and without being able to take within-subject variation into account, it's hard to conclude that differences are truly due to the conditions themselves.

If comparison across conditions is necessary, I would probably do the t-tests described above and then a naïve comparison of the measured pre/post differences for those conditions which met your significance threshold (something like: a1 saw an average difference of 5 measured units from pre to post while c1 saw an average difference of 3 measured units, both significant at alpha=0.05).

This may not be the answer you are hoping for, but the quality and nature of data always puts a ceiling on how much that data can tell you. I could easily be making a wrong assumption (or assumptions) about your data and your study design, but with stats it pays to be conservative when in doubt.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.