# How do I run cross validation on a decision tree in an uplift model?

I have this model from the uplift package,

mod.RF <- upliftRF(as.numeric(response) ~.
+trt(group),
data = df,
mtry = 3,
ntree = 100,
split_method = "KL",
minsplit = 200,
interaction.depth = 1, #also try with id = 2
verbose = TRUE)


I want to make predictions and evaluate those predictions. I'd like to create a test set with 10-fold cross validation. I know how to do that in caret when that it being used to make a model.

How do I evaluate the tree using 10-fold cross validation outside of caret?

Put another way - how can I make new data that I can fill in for the newdata object in predict()and can I use cross validation to do that?

This model is not a decision tree, it is a random forest with ntree = 100 trees.

Here is one general way to do cross validation with the modelr package.

library("tidyverse")
library("modelr")
library("uplift")

# Use example in upliftRF's documentation
set.seed(123)
dd <- sim_pte(n = 1000, p = 20, rho = 0, sigma =  sqrt(2), beta.den = 4)
dd$$treat <- ifelse(dd$$treat == 1, 1, 0)

### fit uplift random forest

uplift_model <- function(data) {
# It is easier to write a separte "fit" function when
# there are many parameters
upliftRF(y ~ X1 + X2 + X3 + X4 + X5 + X6 + trt(treat),
data = data,
mtry = 3,
ntree = 100,
split_method = "KL",
minsplit = 200,
verbose = TRUE)
}
predict_uplift <- function(mod, newdata) {
# For some reason, necessary with predict.upliftRF
newdata <- as_tibble(newdata)
predict(mod, newdata)
}

# 5-fold cross validation
folds <- crossv_kfold(dd, 3)

folds %>%
mutate(model = map(train, uplift_model)) %>%
mutate(pred = map2(model, test, predict_uplift))
#> # A tibble: 3 x 5
#>   train          test           .id   model          pred
#>   <list>         <list>         <chr> <list>         <list>
#> 1 <S3: resample> <S3: resample> 1     <S3: upliftRF> <dbl [334 x 2]>
#> 2 <S3: resample> <S3: resample> 2     <S3: upliftRF> <dbl [333 x 2]>
#> 3 <S3: resample> <S3: resample> 3     <S3: upliftRF> <dbl [333 x 2]>


Created on 2019-03-28 by the reprex package (v0.2.1)