# Significance of time course difference over interval

I am dealing with some data which plotted looks like:

I would like to add a level of significance to my claim that the values of the two time courses are different over the latter half of the experiment (data is sampled at 60 Hz, standard errors of the mean are shaded).

I have tried two approaches:

• Compare per-interval means over this interval - got $p = 2.8 \times 10^{-1}$
• Compare all values in the interval - got $p = 5.6 \times 10^{-53}$

Now, the first approach tells me that my claim is pretty much not significantly supported by my data. The second one tells me that it is - but the p value I get is ridiculously low, though the approach seems sensible and I would not call it inflated.

Could you please tell me what you think is better or how you would advise me to go about this? I have tried fitting a mixed model to this, but the nonlinear time course makes this a bit difficult, and it will surely take me some time to test multiple models with multiple nonlinear components.

I plan to do this eventually, but I am currently looking for a somewhat faster (even if less accurate) approach.

While other answers are more all-encompassing and go into details on linear modelling (which is the better way to do this). I would like to answer the actual, initial question regarding the simplest way to do this.

The best simple alternative, as initially stated, is to compare per-interval means. This means taking the mean value from the first half of time points and creating a per-participant mean, and doing the same thing for the latter half.

Given that operation you obtain as many values as you had participants for each of the 2 new (artificially discretized!) categories. You can now run a related sample t-test, as mentioned in the initial post.

Done!

Of course, you can choose as many discrete categories as you want and use a repeated measurement ANOVA if you are analyzing more than two. The more categories you have, the more autocorrelation (which is what inflated the values of the second approach proposed in the question) becomes a problem again. Regardless of the number of categories you use, you are losing power - since you are using a discrete variable to model a continuous one. This approach is used by many psychologists and is inaccurate, but it is accepted by most - if not all - reviewers.

I'm not convinced this will work, but maybe it will stimulate someone who knows more R/statistics to provide a better answer.

This code assumes you've created a new time variable called "time" which is centered on the time-point dividing periods:

attach(data)
time0 <- time*I(time>=0)
library(nlme)
model1 <- lme(outcome ~ time + group*time0, random= ~ | id)
model2 <- lme(outcome ~ time + group*time0, random= ~ time + time0 | id)
summary(model1)
summary(model2)


or

model3=lm(outcome ~ time + group*time0)
summary(model3)


or

library(rms)
model4=ols(outcome ~ time + group*time0, x=TRUE)
robcov(model4)


The interaction term is meant to test if the slopes are different after the mid-point. http://www.hsph.harvard.edu/fitzmaur/ala2e/ r-code section 8.8 might be interesting. fat.dta and cd4.dta

or you could just restrict your data to the second half:

attach(data)
library(nlme)
model5 <- lme(outcome ~ time*group, random= ~ | id)
model6 <- lme(outcome ~ time*group, random= ~ time | id)
summary(model5)
summary(model6)


and include hybrid model with possible sequential correlations

 model7 <- lme(outcome ~ time*group, random= ~ 1 | id,
+    corr=corCAR1(, form= ~ time | id))
summary(model7)

model8 <- lme(outcome ~ time*group, random= ~ 1 | id,
+    corr=corSymm(, form= ~ time | id))
summary(model8)


Does this account for autocorrelation? This is essentially asking if the correlation structure has been correctly specified. That is something I've never been comfortable with in mixed-models. There are a huge number of options (hence all the commands above), and you may not have chosen the correct correlation structure of the options above. That is what drives people to GEE models, where there are less options and results theoretically more robust to choosing the wrong option.

• What is I in time*I(time>=0) ? Commented Dec 3, 2013 at 21:47
• also, according to this logic, can't I just slice my array at t=2s ? Commented Dec 3, 2013 at 21:49
• just R-code to generate the new time variable such that time0 = 0 if time<0 or time0=time if time>=0. ----- Normally I don't center variable, so I use time and time0=0 if less than intervention period and time0=time-interventiontime if greater than intervention period. results will be the same. So if time=2.2 then time0=0.1 and time time=1 then time0=0 assuming intervention at time2.1. Just don't know how to easily code that in R ---- Commented Dec 3, 2013 at 21:53
• this code assumes you've created a time variable (time) centered on t=2 or whatever you want as your mid-point Commented Dec 3, 2013 at 21:54
• Won't I get the exact same result if I just slice my array at t=2 and analyse the 2nd half? Commented Dec 3, 2013 at 21:56