I have a classifier that is fixed and wish to evaluate its predictive performance using a test dataset. I'm familiar with the situation (e.g. in k-fold CV) where the data is split and the classifier trained on the subsets, but in this case I can't do any training. In particular I'd like confidence intervals on the F1 measure so am planning to bootstrap using the test dataset, calculating F1 with each bootstrap sample.
I can't find any info on this type of scenario - is the bootstrap a valid approach?