5
$\begingroup$

A multilevel model, with one explanatory variable at the individual level (X) and one explanatory variable at the group level (Z):

$$Y_{ij}=\gamma_{00}+\gamma_{10}X_{ij}+\gamma_{01}Z_{j}+\gamma_{11}X_{ij}Z_{j}+u_{0j}+u_{1j}X_{ij}+e_{ij}$$

correlation between $u_{0j}$ and $u_{1j}$ is $0$ . In This pdf , it is written in p.90 that

The interaction effect of these simulated characteristics are presented in table 4. Tested with a blockwise Bonferroni correction, none of the interactions were statistically significant .

But I found all fixed effect $(\gamma_{00},\gamma_{10},\gamma_{01},\gamma_{11})$ and all random effects $(u_{0j},u_{1j})$ statistically significant except individuals-level residual$(e_{ij})$ .

Now my question is if all of them are insignificant according to the mentioned paper , how can the model be valid ? By indicating all of those insignificant , what do they imply ?

Any help is appreciated. Thanks .

EDIT :

I found all fixed effect $(\gamma_{00},\gamma_{10},\gamma_{01},\gamma_{11})$ and all random effects $(u_{0j},u_{1j})$ statistically significant according to this post and the codes are copied here :

simfun <- function(J,n_j,g00,g10,g01,g11,sig2_0,sig01,sig2_1){
     N <- sum(rep(n_j,J))  
     x <- rnorm(N)         
     z <- rnorm(J)         
     mu <- c(0,0)
     sig <- matrix(c(sig2_0,sig01,sig01,sig2_1),ncol=2)
     u   <- MASS::mvrnorm(J,mu=mu,Sigma=sig)

     b_0j <- g00 + g01*z + u[,1]
     b_1j <- g10 + g11*z + u[,2]
      y <- rep(b_0j,each=n_j)+rep(b_1j,each=n_j)*x + rnorm(N,0,sqrt(0.5))
     sim_data <- data.frame(Y=y,X=x,Z=rep(z,each=n_j),group=rep(1:J,each=n_j))
  } 

fit <- function(J,n_j,g00,g10,g01,g11,sig2_0,sig01,sig2_1){
    dat <- simfun(J,n_j,g00,g10,g01,g11,sig2_0,sig01,sig2_1)
    full <- lmer(Y~X+Z+X:Z+(X||group),data=dat,control=lmerControl(optCtrl=list(maxfun=20000)))

  #Testing significance of random intercept
  null.U0 <- update(full, .~.-(1 | group))
  dev.U0 <- as.numeric(2*(logLik(full)-logLik(null.U0)))
  p.U0 <- 0.5*(1-pchisq(dev.U0,1))*(1/6) #multiplied by (1/6) since there are 6 parameters to be tested, that is there are 6 null hypothesis.

 #Testing significance of random slope
  null.U1 <- update(full, .~.-(0 + X | group))
  dev.U1 <- as.numeric(2*(logLik(full)-logLik(null.U1)))
  p.U1 <- 0.5*(1-pchisq(dev.U1,1))*(1/6)

 #Testing significance of intercept of fixed part
 null.int <- update(full, .~.-1)
 dev.int <- as.numeric(2*(logLik(full)-logLik(null.int)))
  p.int <- (1-pchisq(dev.U1,1))*(1/6)

 #Testing significance of X of fixed part
 null.x <- update(full, .~.-X)
 dev.x <- as.numeric(2*(logLik(full)-logLik(null.x)))
  p.x <- (1-pchisq(dev.x,1))*(1/6)

#Testing significance of Z of fixed part
 null.z <- update(full, .~.-X)
 dev.z <- as.numeric(2*(logLik(full)-logLik(null.z)))
  p.z <- (1-pchisq(dev.z,1))*(1/6)

#Testing significance of interaction part
 null.xz <- update(full, .~.-X:Z)
 dev.xz <- as.numeric(2*(logLik(full)-logLik(null.xz)))
  p.xz <- (1-pchisq(dev.xz,1))*(1/6)

 pvals <- data.frame(p.U0=p.U0,p.U1=p.U1,p.int=p.int,p.x=p.x,p.z=p.z,p.xz=p.xz)
}

c1 <- replicate(1000,fit(30,5,1,.3,.3,.3,(1/18),0,(1/18)))
colMeans(apply(c1,2,unlist))
$\endgroup$
2
  • $\begingroup$ "Bu I found all fixed effect (γ 00 ,γ 10 ,γ 01 ,γ 11 ) and all random effects (u 0j ,u 1j ) statistically significant." Maybe you can post how you find this. $\endgroup$
    – Deep North
    Aug 9, 2015 at 10:03
  • $\begingroup$ I found them statistically significant according to this post stats.stackexchange.com/questions/166298/… $\endgroup$
    – user81411
    Aug 10, 2015 at 14:59

1 Answer 1

3
+50
$\begingroup$

There are many questions here, some are a bit confusing. But to get started, and answer your previous post, Bonferroni correction is applied a posteriori, so that raw p-values for each of the factors are essentially divided by n comparisons (hypotheses). So for the Table 4 example, depending how you define family-wise error rate in the context of this question, and what they meant by "block-wise", it can be by parameter (n=27 comparisons for the 7 parameters) or by parameter-group combination (n=9 for 3 groups sizes) vs. 189 (total number of tests).

Secondly, Table 4 refers to interaction terms, not individual effects themselves, which can still be significant. And perhaps clarification is needed on the response variable, which is "Noncoverage" (a type of misclassification indicator) for the 7 regression parameters, not those parameters themselves.

What the quoted statement means is that Noncoverage for some of the 7 parameters (fixed and random effects) depends on Number of Groups and Group Size, but effect of Number of Groups does not vary across levels of Group Size, and the third factor, Intraclass Correlation, which is not significant in any of the interactions or by itself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.