I've been working on some attribution modeling in R, following the markov chain-based methodology described here.
In order to compare the sensitivity and specificity trade-offs of using models of different markov orders, I'd like to look at a ROC curve. I do not know how to technically implement this, using the dataset produced by the channelattribution and markovchain packages. Specifically, I do not know how to produce a prediction and label dataset to feed into ROCR package. Any pointers in the right direction would be greatly appreciated!
For reference, below is sample code for creating the first order markov model using a randomly generated dataset:
# simulating the "real" data set.seed(354) df2 <- data.frame(client_id = sample(c(1:1000), 5000, replace = TRUE), date = sample(c(1:32), 5000, replace = TRUE), channel = sample(c(0:9), 5000, replace = TRUE, prob = c(0.1, 0.15, 0.05, 0.07, 0.11, 0.07, 0.13, 0.1, 0.06, 0.16))) df2$date <- as.Date(df2$date, origin = "2015-01-01") df2$channel <- paste0('channel_', df2$channel) # aggregating channels to the paths for each customer df2 <- df2 %>% group_by(client_id) %>% summarise(path = paste(channel, collapse = ' > '), # assume that all paths were finished with conversion conv = 1, conv_null = 0) %>% ungroup() # calculating the models (Markov and heuristics) mod2 <- markov_model(df2, var_path = 'path', var_conv = 'conv', var_null = 'conv_null', out_more = TRUE)`