I run a binary logistic regression, with a binary dependent variable and a continuous independent one.
Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. For instance, given a probability threshold, I want to compute out-of-sample accuracy or sensitivity.
One solution would be resampling-based estimators: for instance, the method of validation set. I would split the dataset into a training set and a test set, train the model on the training set, and evaluate the performance (e.g. accuracy or sensitivity) on the test set.
This is a generic approach, which can be used with any estimator. For example, I can use this approach also with SVM or neural networks.
But logistic regression is different in many ways from a (deep) neural network: in particular, we know a lot about the properties of the estimator. For instance, we know the confidence interval for the estimated coefficients, and we know an estimate for the irreducible error in the data.
My question is: are there any analytical solutions (not re-sampling based like validation set or cross-validation) to compute out-of-sample performance metrics like accuracy or sensitivity?
For instance, with least squares linear or polynomial regression, I could train the model on the entire dataset (without splitting train/test) and then get the Leave-One-Out Cross-Validation MSE with the following formula: $$ \text{CV} = \dfrac{1}{n} \sum\limits_{i=1}^n \left( \dfrac{y_i - \hat{y}_i}{1-h_i} \right)^2 $$
(see James, Introduction to Statistical Learning, pag. 180)
Is there a similar formula for the accuracy of logistic regression?
