How do you get lmFuncs functions of the rfe function in caret to do a logistic regression?

Question

I've been experimenting with the rfe function in the caret package to do logistic regression with feature selection. I used the lmFuncs functions with the following rfeContol :

ctrl <- rfeControl(functions = lmFuncs, method = 'cv', rerank=TRUE, saveDetails=TRUE, verbose = TRUE, returnResamp = "all", number=100)

Below is the structure of the rfe call:

fit.rfe=rfe(df.preds,df.depend, metric='RMSE',sizes=c(5,10,15,20), rfeControl=ctrl)

df.preds is a data frame of inputs to the model. df.depend is a vector of 1 or 0 corresponding to each row in df.preds to indicate response.

The resulting model accessed in from the fit object in the rfe object is of class lm and produces predicted values of less than zero and greater than 1 when I use the following code with the predict function:

predict(fit.rfe$fit,df,type='response')

Given I'm expecting this to be a logistic, all predicted values should greater than zero and less than one.

Any help will be appreciated.

Answer 1

lmFuncs is fitting linear regression. Just type lmFuncs$fit to see. Try to rewrite it:

lmFuncs$fit<-function (x, y, first, last, ...){   
     tmp <- as.data.frame(x)   
     tmp$y <- y   
 glm(y ~ ., data = tmp,family=binomial)   
}

Note, that I don't know how to attach <environment: namespace:caret> and what is its meaning. You may try this trick on your data and comment the result.