Possible to optimize for area under the precision-recall curve in glmnet logistic regression? tl;dr with the R glmnet package, is it possible to optimize for the area under the precision-recall curve, rather than the area under the ROC curve?
~~~~~
More details
I am using the glmnet package in R to perform elastic-net penalized logistic regression for binary classification on a severely class unbalanced dataset, using type.measure = 'auc' to optimize the area under the curve (AUC) of the receiver operator characteristic (ROC), during cross-validation to select an elastic-net lambda parameter.
However, with severely imbalanced datasets, it appears that area under the Precision-Recall (PRC) curve may be preferable to ROC AUC; e.g., Saito 2015.
This does not seem to be a type.measure option in cv.glmnet. Has anyone found a way to use glmnet logistic regression with PRC-AUC? If not, how important do people think it is to use PRC and not ROC for a severely class imbalanced target?
 A: The gold standard objective function is the log-likelihood or penalized log-likelihood.  Statistical models are not built to optimize anything other than that.  Also you need to take time to understand proper accuracy scoring rules.  Once you are done with fitting the model you can compute the concordance probability ($c$-index; AUROC) to quantify pure predictive discrimination.
Note that elastic net has two penalty parameters, and it is worthwhile to bootstrap the whole process to see whether the set of features selected is reproducible.  It may not be.  For that reason, ridge regression offers better predictive performance.
A: It is important to use the Precision Recall AUC as opposed to ROC AUC for imbalanced datasets.  I use glmnet with my logistic regression models, but I do my own cross validation and select the minimum lambda with the maximum PR AUC.  Make sure to stratify the folds to ensure you have at least one member of the smaller class in each fold.  Weight your classes according to the imbalance ratio.  It also helps to create multiple models assuming your cross validation folds are different each time and then average the lambda across all your models to get a final lambda.
