Using the caret package is it possible to obtain confusion matrices for specific threshold values? I've obtained a logistic regression model (via train) for a binary response, and I've obtained the logistic confusion matrix via confusionMatrix in caret. It gives me the logistic model confusion matrix, though I'm not sure what threshold is being used to obtain it. How do I obtain the confusion matrix for specific threshold values using confusionMatrix in caret?
 A: There is a pretty easy way, assuming tune <- train(...):
probsTest <- predict(tune, test, type = "prob")
threshold <- 0.5
pred      <- factor( ifelse(probsTest[, "yes"] > threshold, "yes", "no") )
pred      <- relevel(pred, "yes")   # you may or may not need this; I did
confusionMatrix(pred, test$response)

Obviously, you can set threshold to whatever you want to try or pick the "best" one, where best means highest combined specificity and sensitivity:
library(pROC)
probsTrain <- predict(tune, train, type = "prob")
rocCurve   <- roc(response = train$response,
                      predictor = probsTrain[, "yes"],
                      levels = rev(levels(train$response)))
plot(rocCurve, print.thres = "best")

After looking at the example Max posted, I'm not sure if there are some statistical nuances making my approach less desired.
A: Most classification models in R produce both a class prediction and the probabilities for each class. For binary data, in almost every case, the class prediction is based on a 50% probability cutoff. 
glm is the same. With caret, using predict(object, newdata) gives you the predicted class and predict(object, new data, type = "prob") will give you class-specific probabilities (when object is generated by train). 
You can do things differently by defining your own model and applying whatever cutoff that you want. The caret website also has an example that uses resampling to optimize the probability cutoff. 
tl;dr
confusionMatrix uses the predicted classes and thus a 50% probability cutoff
Max
