# What if time variable is not significant in longitudinal analysis, can we remove it in the model?

In my longitudinal data, I firstly build a model with two fixed effects, session.week and sync. The former one is just the time variable.

I actually have two subquestions :

(1) What if time variable is not significant? can I remove it?

When I run the following code with R

  quality.Model.1<- lmer(quality ~  session.week + sync  + (1|group.name/student.id), data = dfSubset)
pvals.fnc(quality.Model.1)


What I get is the following

  $fixed Estimate MCMCmean HPD95lower HPD95upper pMCMC Pr(>|t|) (Intercept) 3.3166 2.9966 0.9498 4.6601 0.0154 0.0000 session.week -0.0458 -0.0265 -0.2369 0.1867 0.7934 0.6352 sync 1.3079 1.6866 0.1026 3.7183 0.0244 0.0206$random
Groups        Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upper
1 student.id:group.name (Intercept)   0.3303     0.1417   0.1535     0.0000     0.3912
2            group.name (Intercept)   0.0000     0.1975   0.3523     0.0000     1.2247
3              Residual               0.6365     0.6798   0.6859     0.5408     0.8552


As you see, the time variable is not significant, but sync is significant (p<.05) If I run a summary of the above model, we got:

  AIC   BIC logLik deviance REMLdev
113.3 124.4 -50.67    96.63   101.3


Actually although this is a longitudinal study, I don't actually expect the time would change my independent variable. The non-significance result actually confirms my expectation. Since the time variable (session.week) is not significant, can I remove it in my model, so it becomes:

 quality.Model.2<- lmer(quality ~  sync  + (1|group.name/student.id), data = dfSubset)
pvals.fnc(quality.Model.2)


The result of this models is :

  $fixed Estimate MCMCmean HPD95lower HPD95upper pMCMC Pr(>|t|) (Intercept) 3.091 2.897 1.5119 3.931 0.0032 0.0000 sync 1.413 1.691 0.3116 3.320 0.0092 0.0074$random
Groups        Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upper
1 student.id:group.name (Intercept)   0.3371     0.1480   0.1583     0.0000     0.3903
2            group.name (Intercept)   0.0000     0.1787   0.2955     0.0000     1.0023
3              Residual               0.6284     0.6721   0.6784     0.5373     0.8353


And if we summary the result, we got:

  AIC BIC logLik deviance REMLdev
108.7 118 -49.35    96.91   98.71


We don't lose the significance of sync, but the AIC has reduced. Can I conclude that my second model (without the time variable) is better? And can I just remove the time variable safely? It just made me feel wired to remove the time variable in a longitudinal study.

(2) What if the the significance of sync is canceled when testing a model with interaction terms?

Suppose we don't give up time variable, and I built a mixed model with an interaction term as follows:

  quality.Model.3<- lmer(quality ~  session.week * sync  + (1|group.name/student.id), data = dfSubset)
pvals.fnc(quality.Model.3)


and we got the following :

  $fixed Estimate MCMCmean HPD95lower HPD95upper pMCMC Pr(>|t|) (Intercept) 4.6887 4.3486 1.058 7.477 0.0200 0.0004 session.week -0.3859 -0.3402 -0.953 0.313 0.2842 0.1744 sync -0.6694 -0.2381 -4.267 3.874 0.8706 0.6835 session.week:sync 0.5110 0.4610 -0.395 1.368 0.2916 0.2018$random
Groups        Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upper
1 student.id:group.name (Intercept)   0.3488     0.1465   0.1576       0.00     0.3941
2            group.name (Intercept)   0.0000     0.1861   0.3387       0.00     1.2614
3              Residual               0.6260     0.6808   0.6869       0.53     0.8444


Nothing is significant at all, including the interaction term!!!, after introducing the interaction term. If we do summary on it:

     AIC   BIC logLik deviance REMLdev
113.7 126.7 -49.85    94.92   99.71


The AIC is larger than both quality.Model.1 and quality.Model.2.

(3)What to choose

So, which model should I choose if I want to see the effect of sync on quality? If I remove the time variable as fixed effects, then it's done perfectly. If I don't remove the time variable, then the main effects model suggests the significance of sync, but the interaction model cancels out everything.

## 1 Answer

I don't think the fact that this is longitudinal data changes any of the usual points about this sort of analysis:

1) If you have an interaction term in your model, you want all the main effects involved in the interaction in your model (with very rare exceptions)

2) We shouldn't choose variables exclusively by whether they are significant - they can also be important for other reasons. Here, the time variable is important because a) It's in an interaction b) It's "natural" - you sort of have to look at the effect of time in a longitudinal analysis c) It affects other parameters.

3) We also shouldn't reject models just because they don't fit our expectations. In your case, it looks like there is an interaction of time and sync.

4) Once there is an interaction, analyzing the main effects is complex. So your statement that the interaction cancels out everything is, while true looking at the output, not really so. The main effects posted when an interaction is included are the effects of that variable when the other variable = 0. So.... you are seeing the effect of time when sync = 0 and the effect of sync when time = 0.

• Thanks Peter. I have two comments : (1)Some one suggests to select the best model based on AIC and BIC, if this is the case, then definitely, the model without time is the best. (2) After introducing the interaction terms, nothing is significant, even the interaction term is not significant. But the main effect is canceled out. In this case (interaction term is not significant), how do you think I should report the result? Should I report the variable sync significantly affects quality? Or, nothing affects quality significantly? – nan Sep 23 '13 at 9:55
• 1) Using AIC or BIC is OK, as long as the model you wind up choosing doesn't violate rules. Including an interaction without the main effects does violate a rule. 2) Whether to include the interaction is up to you - is it interesting? Is it useful? Does it help you understand what is going on? 3) However, deleting time seems to me to be contrary to the spirit of longitudinal analysis. Would you include it only as a random effect? That might be OK. – Peter Flom Sep 23 '13 at 10:18
• (1)I don't get why you say "including an interaction without the main effects". For the three models i built, I never include an interaction without the main effects. The model without time is without interaction terms as well. (2) the design is longitudinal because it is on a course, and we did measurements 4 times during a semester, but our main aim is not to examine the effect of time, but to test the effect of the variable "sync". You think it also make sense to do "quality ~ sync + (1|group.name/student.id) + (1|session.week)", where the time variable is included as a random effect? – nan Sep 23 '13 at 10:30
• and in R, session.week * sync means session.week + sync + session.week:sync. So there is no problem of including an interaction with the main effects. – nan Sep 23 '13 at 11:11
• I don't see model selection as useful here, as opposed to fitting a well-formed full model and interpreting it. – Frank Harrell Sep 23 '13 at 12:05