lme4: profile takes lot of time for a complicated model I have a pretty complicated mixed model (cross classified data) and a pretty large dataset (>700000 obs.). The lmer function took few hours to compute, but profile takes over two days and is still working with no effects. Is there any point in waiting some more time, or does it suggest that something went wrong..? Except the time it consumes I have no other reasons to worry, but it still is strange for me.
Btw, if you could suggest something more on profile function because the lme4 documentation seems a little bit vague I would be grateful.
 A: There is a bit more documentation on the profiling procedure in the JSS preprint posted on ArXiv.
Here's what you should know about the relative speed of profiling:
Suppose that your model has n fixed-effect parameters and m random-effect parameters.


*

*each of m+n parameters, plus (for LMMs) the residual standard deviation, is profiled separately (eventually we'd like to allow these to be computed in parallel)

*for each parameter, on the order of 10-12 optimizations need to be done 

*for GLMMs the optimizations are over n+m-1 dimensions for all parameters

*for LMMs the optimizations are over m-1 dimensions for the random effects  parameters and over m dimensions for the fixed effects parameters


So I can't tell you exactly how your model fit will scale, but it doesn't surprise me that the profiling is an order of magnitude slower.  If you like you could set verbose=1 when profiling to get information for every profiling step ...
For the simple GLMM example in ?glmer, benchmarking on my machine shows that profiling is 46x slower than the original fit.
update: parallel profiling (i.e. fitting profiles for each parameter on a separate worker/core) has been available in lme4 for the last few stable versions (1.1-8), although it's not listed in the NEWS file: see the parallel= argument under ?profile.merMod.
Here's a homemade version of parallel profiling:
library('lme4')
fm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
npars <- length(unlist(getME(fm1,c("theta","fixef"))))
library("parallel")
cl <- makePSOCKcluster(rep("localhost",4))
parallel::clusterExport(cl, varlist="fm1")
res <- parLapply(cl,X=1:npars,fun=CIfun)
do.call(rbind,res)

You might also want to look into using nloptwrap as your optimizer -- it can speed up large lme4 fits considerably.
