1
$\begingroup$

I'm about to start up a project (as a high school teacher) that involves comparing the change in average between two different groups.

It's an experiment, and the setup is the following:

  1. Obtain a population of respondents,
  2. Have each respondent answer a questionnaire that scores them on an index,
  3. Divide the respondents randomly into two groups,
  4. Subject them to different information,
  5. Let them answer the questionnaire again and see if there is any difference in the change between the two groups.

I'm thinking of using the z-test to compare the averages, but I'm not sure exactly how. Any ideas or other suggestions are welcome!

Cheers

EDIT: The statistical approach needs to be as simple as possible, as it's for high school students. The z-test looks implementable in Excel, that's part of the reason why I like that for this study.

There are probably numerous issues with that approach - but think along the lines of the weightless ropes and frictionless surfaces often used in high school physics ;-)

Improve this question
$\endgroup$
3
  • $\begingroup$ Who do you want your results to generalize to? Only the students who took the questionnaire or others that could have taken it? In the first case a z test is justified. In the second case Peters answer is right and either t-test will be fine (they are computationally equivalent and easy to do in excel or by hand). $\endgroup$
    – russellpierce
    Commented Mar 15, 2013 at 15:18
  • $\begingroup$ Best practices for this design have already been discussed on this site, see stats.stackexchange.com/questions/3466/… $\endgroup$
    – Gala
    Commented May 14, 2013 at 15:31
  • $\begingroup$ But obviously the focus was on power, correctness and flexibility, not on doing something easy in Excel. I think that the simplest reasonable approach would be to compute difference scores (score on the questionnaire after the manipulation - score at the beginning) and compare the two groups using a regular independent sample t-test. $\endgroup$
    – Gala
    Commented May 14, 2013 at 15:33

1 Answer 1

Reset to default
1
$\begingroup$

There are several approaches e.g.

  1. ANOVA/Regression/ANCOVA (all the same, really) with the final score as the DV and group and initial score as IV

  2. Paired t-tests (this does not allow other covariates)

  3. ANOVA on change scores

However, if you are still designing this study, I suggest more time points for measurement; this allows more interesting hypotheses. If you take that suggestion, then for analysis I would look at multilevel models.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I am still designing this study, but it needs to be as simple as possible: it's for high school students :-) Anything apart from looking up a test statistic in a table / somewhere else needs to be done in MS Excel or similar. $\endgroup$
    – trolle3000
    Commented Feb 13, 2013 at 13:37
  • 2
    $\begingroup$ OIC.... Well, then I'd go with the paired t-test or (equivalently) a one-sample t-test of the differences in scores. $\endgroup$
    – Peter Flom
    Commented Feb 13, 2013 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.