# Get predictions from mgcv:gam model with a new level in a random effect smooth

Is there a way to get a prediction from an gam model (from package mgcv) that contains random effect smooths, where the new data contains a level of the random effect that didn't exist in the training set? Something comparable to the "re.form=NA" parameter in lme4:predict.merMod ?

library(hflights) # get a big data set
hflights$UniqueCarrier <- factor(hflights$UniqueCarrier) # turn carrier into a factor
newhflights <- head(hflights) # create "new" d.f. for prediction
newhflights$UniqueCarrier <- "ZZ" # change carrier to a previously unseen value library(mgcv) m_gam <- gam(ActualElapsedTime ~ s(Distance) + s(TaxiOut, UniqueCarrier, bs="re"), data=hflights) predict(m_gam, newdata=newhflights) # fails with "ZZ not in original fit" library(lme4) m_lmer <- lmer(ActualElapsedTime ~ Distance + (TaxiOut | UniqueCarrier), data=hflights) predict(m_lmer, newdata=newhflights) # fails with "new levels detected in newdata" predict(m_lmer, newdata=newhflights, re.form=NA) # works  I have searched for an answer and found this question, but it didn't appear to be directly related to my problem. Predicting with random effects in mgcv gam But I did try adding a dummy variable to the model based on it. hflights$dummy <- 1
newhflights\$dummy <- 0
m_gam <- gam(ActualElapsedTime ~ s(Distance) + s(TaxiOut, UniqueCarrier, bs="re", by=dummy), data=hflights)
predict(m_gam, newdata=newhflights) # still doesn't work, not in original fit


I'm out of ideas. Am I overlooking something obvious?

• I suppose refitting the model with ZZ added as a level to the original data is out of the question? (you'd need to refit with drop.unused.levels = FALSE). Alternatively, just set that new level to be one of the existing levels; as long as you've turned off the re term it will make no difference to the predictions (assuming the models are as per the ones shown). – Gavin Simpson Jun 12 '15 at 20:16
• The new previously unseen levels are not necessarily known until prediction time, and refitting the model is not a desirable option. Overwriting the random effect level with a known one (then changing it back post-prediction) seems to work, and I guess that's the only real option. It seemed awkward to do it that way, so I wondered if there was a more elegant solution. Thanks! – colonel.triq Jun 15 '15 at 21:42
• No, even Simon Wood freely admits that handling of random effects is simple at best in mgcv::gam. Perhaps mgcv::gamm or gamm4::gamm might work better for you seeing as the spline fixed-effects part of your model appears relatively simple. – Gavin Simpson Jun 16 '15 at 2:17