11
$\begingroup$

I am currently arguing with someoe on how to correctly treat data with multiple measurements for each subject. In this case data was gathered for each subject within a short time for different conditions within each subject. All measurements gather exactely the same variable, just multiple.

One option now is to just group the data by conditions and not care that multiple data points come from one subject. However the data points from each subject are probably not completely independent.

The other alternative is, to first take the mean of all measurements for each condition from each subject and then compare the means. However this will probably impact the significance, since in the final analysis it is not taken into account, that the means have less error.

How can you correctly analyse such data? Is this somehow taken care of in SPSS? In principle it should be possible to calculate the error margin when calculating a mean and than considering this in the final analysis, but I do not guess that SPSS is somehow doing this calculation behind my back.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Is this a repeated measures design such that each subject runs in all, or many, of the conditions? Or, is it just an independent groups, or measures, design where each subject is in one condition? $\endgroup$
    – John
    Commented Mar 14, 2012 at 0:45
  • $\begingroup$ In this design each subject runs in all conditions. However there are some data points, which have to be rejected, because subjects failed at the task at hand. It is unlikely that a subject will fail at all subtasks for a single condition (there are about 40 repetitions per condition), so most likely each subject will have data points for all conditions. $\endgroup$
    – LiKao
    Commented Mar 14, 2012 at 7:32

2 Answers 2

Reset to default
10
$\begingroup$

It would be a violation of independence to "group the data by conditions and not care that multiple data points come from one subject". So that is a no go. One approach is to "to take the mean of all measurements for each condition from each subject and then compare the means". You could do it that way, you wouldn't violate independence, but you are losing some information in the aggregation to subject level means.

On the face of it, this sounds like a mixed design with conditions between subjects and multiple time periods measured within subjects. However, that raises the question, why did you collect data at multiple time points? Is the effect of time, or the progression of a variable over time expected to be different between conditions? If the answer is yes to either of those questions, then given the structure of the data, I would expect that what you are interested in is a mixed ANOVA. The mixed ANOVA will partition the subject variance out of the SSTotal "behind your back" as it were. But whether that partitioning helps out your between subjects test of conditions depends on several other factors.

Anyway, in SPSS/PASW 18 Analyze -> General Linear Model -> Repeated Measures. You'll have one row for each subject and one column for each time point as well as one as their condition identifier. The condition identifier will go into the "between" section and the repeated measures will be taken care of when you define the repeated measure factor.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Ok, this is what I have thought. The multiple data points per condition are collected for two reasons. One is that the data should be more reliable this way. The other reason is, that some data points have to be discarded (subjects did not follow instruction correctly all times). The conditions are completly within subjects, so we do not have a mixed desing in this case at all. Unfortunately a repeated measure is out of question, since we have about 40 repetitions per condition in each subject. The high number of repetitions however means, we loose a lot of information when using mean. $\endgroup$
    – LiKao
    Commented Mar 14, 2012 at 7:38
  • $\begingroup$ Then I recommend John's answer. A mixed model is likely preferable. That can model both the mean and the variability within each subject and respect nesting. One issue with such an analysis is that the 'correct' degrees of freedom are unclear and thus the thresholds for statistical significance are also unclear. In contrast to John's provided code, I would recommend fitting a random slope for your condition effect (different subjects display different effects). I've seen some simulations that suggest failing to do so may elevate your Type I error rate. $\endgroup$
    – russellpierce
    Commented Mar 14, 2012 at 22:02
5
$\begingroup$

Repeated measures design is the traditional way to handle this, as drknexus mentions. When doing that kind of analysis you have to aggregate to one score/condition/subject. It's sensitive to violations of assumptions of sphericity and other issues. However, the more modern technique is to use multi-level modelling or linear mixed effects. Using this technique you do not aggregate the data. There are several treatments of this available but I don't currently know the best basic tutorial. Baayen (2008) Chapter 7 is good. Pinheiro & Bates (2000) is very good but from the sounds of things follow their advice in the intro and read the bits recommended for beginners.

If you want to just get an ANOVA style result, assuming all of your data are in long format (one line / data point) and you have columns indicating subject, response (y), and a condition variable (x), you could try looking at something like this in R (make sure the lme4 package is installed).

library(lme4)
dat <- read.table('myGreatData.txt', header = TRUE)
m <- lmer( y ~ x + (1|subject), data = dat)
summary(m)
anova(m)

You could of course have many more conditions variable columns, perhaps interacting. Then you might change the lmer command to something like...

m <- lmer( y ~ x1 * x2 + (1|subject), data = dat)

(BTW, I believe that not aggregating in repeated measures in order to increase power is a formal fallacy. Anyone remember the name?)

Improve this answer
$\endgroup$
1
  • $\begingroup$ I think the fallacy of failing to aggregate and using the df from the number of responses rather than the number of subjects is a violation of independence. Alternatively, (I think) it might be thought of making an inference at the level of individual item responses for a fixed set of subjects. $\endgroup$
    – russellpierce
    Commented Mar 14, 2012 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.