3
$\begingroup$

I am doubting myself on which analysis to run for the following: 18 participants were evaluated at 4 time points with different conditions at each time. They were given scores (on a discrete visual analog scale) by 2 raters.

The scores were calculated for a pair of participants: the pairs changed at each time point. I do know which participant comprises each pair.

Is that a 2-way repeated measures ANOVA? Some variation of Friedman test?

$\endgroup$
  • 2
    $\begingroup$ For clarity, did the same two raters see all 18 participants for all four time points? Did all 18 participants experience the same four-conditions? Finally, what is your research question: rater difference, condition difference, or time difference? $\endgroup$ – doug.numbers Apr 20 '13 at 21:16
  • $\begingroup$ Yes. Both raters reviewed all 18 participants at the 4 points. I am interested in condition difference (perhaps controlling for practice effect over time), but using the raters to confirm scoring. THANKS!!! $\endgroup$ – Patrick Apr 22 '13 at 0:24
  • $\begingroup$ Try to understanding your data before you do an ANOVA. First, plot your data to ensure that it is normally distributed. Then plot total score by rater to determine if there is a difference between raters. Estimate mean and 95% CI for the mean for each rater. Then estimate mean and 95% CI for the mean for the average total score of the raters for each of the four conditions. Check and see conditions don't have overlapping of the confidence intervals. Try doing boxplots with notches to visualize overlap. ANOVA will tell you if one of the conditions is different, but not which. $\endgroup$ – doug.numbers Apr 22 '13 at 12:55
1
$\begingroup$

Try to understanding your data before you do an ANOVA. First, plot your data to ensure that it is normally distributed. Then plot total score by rater to determine if there is a difference between raters. Estimate mean and 95% CI for the mean for each rater. Then estimate mean and 95% CI for the mean for the average total score of the raters for each of the four conditions. Check and see conditions don't have overlapping of the confidence intervals. Try doing boxplots with notches to visualize overlap. ANOVA will tell you if one of the conditions is different, but not which.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

If each rater gave 9 ratings -- one for each pair of subjects -- at each time point, if the subjects were re-paired at each time point, and if it is not known who was paired with whom, then I don't think there is any proper way to analyze the data, because of the unknown and unestimatable correlations among the ratings from from one time to the next.

If you are willing to treat the data as if there were 18 different subjects at each time point (72 subjects total) then you could do a 1-between, 1-within analysis, where "between" and "within" are relative to the pairs, which would play the role that is usually played by subjects in such analyses: pairs are nested within time and crossed with raters; time is the between-pairs (grouping) factor -- to check for a time difference, you must look at different pairs -- and raters is the within-pair (repeated measures) factor -- you can check for a rater difference within each pair.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.