Different t-value for the same data in R (nlme vs lme4 package)

Question

I have two dependent variable Prime Type (five levels), and Prime Relatedness (two levels), and one dependent variable; Reaction Time (RT). I have ran a linear mixed-effect model in R using the following formula for lme4 package:

data.mod1=lmer(RT~Related*PrimeType*(1|Subject)+(1|Item),mydata)

Then I ran a similar formula for nlme package

data.mod1.lme=lme(RT~Related*PrimeType, random=~1|Subject/Item,mydata)

Considering that these models are analyzing the same data set using linear mixed effect models, the t-test show different values! In fact, for one of the variables it shows a significant p-value in nlme model but not in lme4 package:

nlme.variable t=-1.98707 p-value=0.0470
lme4.variable t=-1.14    p-value=0.2917866* 

The p-value in lme4 is not calculated and sampMCMC is no longer available, I had to calculated using the following formula 2(1 - pt(abs(x), df)) --> 2*(1 - pt(abs(-1.14), 7))

My question is that why are the t-values differ between the two packages. Do I need to change or add something in R to have both models show the same t-values? Also, is my calculation of the p-value wrong? If yes, what is the correct way for it to be calculated it?

EDIT: nlme experts - what is the correct code for the above mentioned formula to add cross random effect?

Answer 1

You haven't given a reproducible example, but:

library(lme4)
library(nlme)
library(pbkrtest)

fm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
## see definition of KRSumFun below, taken from ?drop1.merMod:
## requires recent [development?] version of `lme4`:

packageVersion("lme4")  ## 1.1.2
drop1(fm1,test="user",sumFun=KRSumFun)

## Model: Reaction ~ Days + (Days | Subject)
## Method: Kenward-Roger via pbkrtest package
## 
## 
##        ndf ddf  Fstat    p.value F.scaling
## <none>                                    
## Days     1  17 45.853 3.2638e-06         1

fm2 <- lme(Reaction ~ Days , random = ~Days | Subject, sleepstudy)
anova(fm2)
##             numDF denDF   F-value p-value
##(Intercept)      1   161 1454.0766  <.0001
## Days            1   161   45.8534  <.0001

Note that this is a case (random-slopes model) where lme actually gets the wrong answer for the denominator degrees of freedom.

KRSumFun <- function(object, objectDrop, ...) {
       krnames <- c("ndf","ddf","Fstat","p.value","F.scaling")
       r <- if (missing(objectDrop)) {
           setNames(rep(NA,5),krnames)
       } else {
          krtest <- KRmodcomp(object,objectDrop)
          unlist(krtest$stats[krnames])
       }
       attr(r,"method") <- c("Kenward-Roger via pbkrtest package")
       r
    }

Answer 2

I primarily use lme4, so you'll have to excuse me if this answer is incorrect. I believe you specified crossed-random effects in the lmer function call, but nested random effects in the lme function call. I'm not sure how to do crossed-random effects in lme, but perhaps this or this might yield some insight.

One other thing (unrelated), is that RTs usually possess a distribution with positive skew and could benefit from an inverse transformation. See my answer here for a little more info.