Why lambda (regularization paramter) in predict can be different from lambda for fitting model?

Question

lambda is the parameter of regularization. I'm talking in the context of glmnet, and this may be a question generally applied for any model with regularization.

I though glmnet calculates the coefficients in the model based on the lambda sequence for fitting model. So the result doesn't have the model for other lambda values. But why I can still do prediction using other lambda values?

library(glmnet)
library(ISLR)

data(Caravan)

y = dta$Purchase
x = as.matrix( dta[ , -which(colnames(dta)=='Purchase') ] )

lam1 = 10^(-10:-1)

lr = glmnet(x, y, family='binomial', lambda = lam1)
lr
preds = predict(lr, x, lam1*3, type = "response") # prediction using other lambda

---

I ask this because of this:

I roughly know that lambda, when not given by user, is generated in glmnet based on the data.

cv.glmnet firstly run glmnet for all the data (I call it model 0), and then run glmnet for each folder (for a k-folder cv, it's k times, and I call them model 1-k).

The purpose to run glmnet for all the data (model 0) is to get the lambda sequence (call it lambda-0 seq) to be used in cv [see P5 in 1].

Then cv.lognet is called and it calculates the cv, using lambda-0 seq. First, the values in lambda-0 seq that don't work with model 1-k are ruled out. Prediction is then made using model 1-k and lambda-0 seq.

Relevant code in cv.glmnet

glmnet.object = glmnet(x, y, weights = weights, offset = offset, lambda = lambda, ...)
lambda = glmnet.object$lambda
# has run glmnet once to get the lambda seq, and now use it for cv

fun = paste("cv", class(glmnet.object)[[1]], sep = ".")  # here fun = cv.lognet
cvstuff = do.call(fun, list(outlist, lambda, x, y, weights, offset, foldid, type.measure, grouped, keep))
# outlist is model 1-k

and cv.lognet: (See the source code of these functions for more please.)

mlami = max( sapply( outlist, function(obj) min(obj$lambda) ) )
# ignore lambda doesn't work for every subset
which_lam = lambda >= mlami

#prediction for each folder
fitobj = outlist[[i]]
preds = predict(fitobj, x[which, , drop = F], s = lambda[which_lam], offset = off_sub, type = "response") #use lambda-0 seq for pred in model 1-k

My question is why lamdba-0 seq can be passed to model 1-k to do prediction? Model 1-k should only have models (coefficients) at their only lambda sequence. Any help's appreciated. Thanks

Answer 1

In your case, the package can produce another fit of the model or use linear interpolation.

From the documentation of predict.glmnet:

If exact=TRUE, and predictions are to made at values of s not included in the 
original fit, these values of s are merged with object$lambda, and the model
is refit before predictions are made. If exact=FALSE (default), then the predict function 
uses linear interpolation to make predictions for values of s that do not
coincide with those used in the fitting algorithm. Note that exact=TRUE is fragile
when used inside a nested sequence of function calls. predict.glmnet() needs
to update the model, and expects the data used to create it to be around.