0
$\begingroup$

I have this model from the uplift package,

mod.RF <- upliftRF(as.numeric(response) ~.
                   +trt(group), 
                   data = df,
                   mtry = 3,
                   ntree = 100,
                   split_method = "KL",
                   minsplit = 200,
                   interaction.depth = 1, #also try with id = 2
                   verbose = TRUE)

I want to make predictions and evaluate those predictions. I'd like to create a test set with 10-fold cross validation. I know how to do that in caret when that it being used to make a model.

How do I evaluate the tree using 10-fold cross validation outside of caret?

Put another way - how can I make new data that I can fill in for the newdata object in predict()and can I use cross validation to do that?

$\endgroup$
0
$\begingroup$

This model is not a decision tree, it is a random forest with ntree = 100 trees.

Here is one general way to do cross validation with the modelr package.

library("tidyverse")
library("modelr")
library("uplift")

# Use example in upliftRF's documentation
set.seed(123)
dd <- sim_pte(n = 1000, p = 20, rho = 0, sigma =  sqrt(2), beta.den = 4)
dd$treat <- ifelse(dd$treat == 1, 1, 0)

### fit uplift random forest

uplift_model <- function(data) {
  # It is easier to write a separte "fit" function when
  # there are many parameters
  upliftRF(y ~ X1 + X2 + X3 + X4 + X5 + X6 + trt(treat),
           data = data,
           mtry = 3,
           ntree = 100,
           split_method = "KL",
           minsplit = 200,
           verbose = TRUE)
}
predict_uplift <- function(mod, newdata) {
  # For some reason, necessary with `predict.upliftRF`
  newdata <- as_tibble(newdata)
  predict(mod, newdata)
}

# 5-fold cross validation
folds <- crossv_kfold(dd, 3)

folds %>%
  mutate(model = map(train, uplift_model)) %>%
  mutate(pred = map2(model, test, predict_uplift))
#> # A tibble: 3 x 5
#>   train          test           .id   model          pred           
#>   <list>         <list>         <chr> <list>         <list>         
#> 1 <S3: resample> <S3: resample> 1     <S3: upliftRF> <dbl [334 x 2]>
#> 2 <S3: resample> <S3: resample> 2     <S3: upliftRF> <dbl [333 x 2]>
#> 3 <S3: resample> <S3: resample> 3     <S3: upliftRF> <dbl [333 x 2]>

Created on 2019-03-28 by the reprex package (v0.2.1)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.