1
$\begingroup$

I am planning an evaluation which will deploy pre- and post-tests for one group of students. I would like to perform a dependent t-test but may not be able to perform a repeated measures analysis since we may not be able to collect full names, maybe only ages from the students.

Is there any meaningful difference between a matched pairs (e.g., pairing by ages) versus repeated measures analysis? My preference would be to perform a repeated measures analysis if we could ask the students to enter their full names. If we can't collect names and have to do a matched pairs analysis using ages, would there be any disadvantage in doing so?

Thanks in advance for any feedback.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Your question is really unclear.

  1. How many measures (i.e., tests of students) do you intend to deploy? Two? More than two?
  2. Are the repeated measures able to be matched up for each student? That is to say, can you somehow collect tests in a way that you can tell that a pair (or group if the answer to item 1 above is more than two) of tests were completed by the same student? Can you anonymize them with an ID?
  3. What exactly do you mean by "pair by age?" In a typical student population, you should expect many students to be of the same age. What happens if you have 5 students who are all the same age? How do you "pair up" their repeated measures? Are you suggesting you take the means of each measure in time for each observed age, and do a repeated measures test on each age? This is obviously problematic.
  4. What is the hypothesis you wish to test or the inference you wish to make?

What I can say is that you cannot do a matched pairs or repeated measures test if you can only identify pairs/groups of tests by age. The assumption for such tests requires that you can identify which observations are repeated measures on the same experimental unit.

Improve this answer
$\endgroup$
9
  • $\begingroup$ 1) We are planning to deploy one pre-test, one post-test immediately after the intervention, and a second post-test 2-months after the intervention. 2) I agree that assigning an ID to each student would work but this approach may complicate the evaluation (which we are really trying to avoid). Without the students' names, we would not be able to track which scores are for which students. 3) That is true. So I could not randomly pair the students with the same ages, even if many students have the same age? (e.g., randomly pair two 7 yr. olds, until all 7 yr. olds have been paired)? $\endgroup$
    – Lucas
    Commented Nov 25, 2015 at 23:38
  • $\begingroup$ ... 4) In the previous phase of the evaluation, I was able to perform a t-test for the pre-test and post-test scores since I had the students names, ages, locations, etc. I would like to be able to perform a t-test or similar analysis again but this time I may not be able to match the pre-test to post-test scores to any one student. So I guess my questions is, what alternatives to a t-test for repeated measures are there to determine if any changes in mean test scores are statistically significantly different between pre- and post-tests? $\endgroup$
    – Lucas
    Commented Nov 25, 2015 at 23:46
  • $\begingroup$ No. You can't randomly pair them up. Paired or repeated measures tests assume that the test statistic follows a distribution corresponding to paired or matched observations. Randomly pairing them will violate that assumption and will lead to an incorrect inference. Don't do it. $\endgroup$
    – heropup
    Commented Nov 25, 2015 at 23:54
  • $\begingroup$ Just do a regular analysis of variance, treating each repeated measure as independent, because there is no way to associate them longitudinally. Because you have three measurements instead of two, you also can't use a t-test, whether paired or two-sample independent. $\endgroup$
    – heropup
    Commented Nov 25, 2015 at 23:57
  • $\begingroup$ Thank you for those clarifications and suggestions. I will look into AoV. Could not I still perform several t-tests, for example one for any changes between the pre-test and immediate post-test, another between the pre-test and 2-month post-test, etc.? $\endgroup$
    – Lucas
    Commented Nov 26, 2015 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.