I'm aiming to use caret::sbf to filter a large number of predictors before using different machine learning models to predict a binary outcome. I would like to filter for variables that are identified as significant in at least 70% of LGOCV instances. However, I am unsure how to articulate this in the caretSBF score function. Below is an example of how it might work.
svmSBF <- caretSBF svmSBF$summary <- function(...) c(twoClassSummary(...), defaultSummary(...)) #svmSBF$score <- ?? svmSBF$filter <- function(score, x, y) score > 70 data <- twoClassSim(n = 100, linearVars = 300) fit <- sbf( form = Class ~ ., data = data, method = "svmLinear", tuneGrid=expand.grid(C = 2^c(seq(-25,10,.1))), preProc = c("center", "scale"), trControl = trainControl(method = "repeatedcv", number = 10, repeats = 10, classProbs = TRUE, savePredictions = TRUE), sbfControl = sbfControl(method = "LGOCV", number = 100, p = .8, functions = svmSBF, saveDetails = TRUE))
Any help would be greatly appreciated.