0
$\begingroup$

I am trying to perform model explainability for the best performing model using LIME for a classification problem. The y variable is whether a tumour is malignant or benign. Same question as (https://stackoverflow.com/questions/75867990/using-lime-in-r-to-explain-the-best-performing-model)

This is my attempt below:

models<-c("svmRadial","rf","knn")
results_table <- data.frame(models = models, stringsAsFactors = F)

for (i in models){
  model_train <- train(class~., data = training, method = i,
                       trControl= control, metric = "Accuracy")
  assign("fit", model_train)
  predictions <- predict(model_train, newdata = testing)
  table_mat<-table(testing$class, predictions)
  accuracy<-sum(diag(table_mat))/sum(table_mat)
  precision_ <-posPredValue(predictions, testing$class)
  recall_ <- sensitivity(predictions, testing$class)
  
  # put that in the results table
  results_table[results_table$models %in% i, "Accuracy"] <- accuracy
  results_table[results_table$models %in% i, "Precision"] <- precision_
  results_table[results_table$models %in% i, "Recall"] <- recall_
 
}

For this I got the following results, which are on results_table:

Model        Accuracy   Precision  Recall
svmRadial    0.9588235  0.9814815  0.954955
rf           0.9705882  0.9732143  0.981982
knn          0.9705882  0.9732143  0.981982

I have used LIME and it has worked (my attempt is below), but now I do not know which one of the models it is explaining. How do I know which model it is explaining? Is is explaining all three models or just the first model.

library(lime)
explainer_caret <- lime(training,  model_train)
 
explanation <- explain(testing[15:20, ], explainer_caret,
                       labels="malignant",
                       n_permutations=5,
                       dist_fun="manhattan",
                       kernel_width = 3,
                       n_features = 5)

enter image description here

$\endgroup$

1 Answer 1

0
$\begingroup$

It explains model_train. In your loop, you overwrite model_train three times (more precisely, you create it once and overwrite it twice). The last overwriting results in a model trained using method="knn", and that is what is explained.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.