# Evaluating a fixed classifier

I have a classifier that is fixed and wish to evaluate its predictive performance using a test dataset. I'm familiar with the situation (e.g. in k-fold CV) where the data is split and the classifier trained on the subsets, but in this case I can't do any training. In particular I'd like confidence intervals on the F1 measure so am planning to bootstrap using the test dataset, calculating F1 with each bootstrap sample.

I can't find any info on this type of scenario - is the bootstrap a valid approach?

Also because the model is fixed I think that it can be seen simply as a function of input. If the input is a random variable $X = X_1,X_2,..,X_n$, than our model is also a random variable $M = (M_1,M_2,..,M_n) = f(X) = f(X_1),f(X_2),..,f(X_n)$. We go further and say that F-measure $F_1$ is a function of some $M_i$ random variables. The point is that we can think of $F_1$ as a random variable which has an unknown distribution, but based on the transformations through the classifier we can obtain a sample for this $F_1$ random variable. Thus percentile bootstrapping looks to me a valid way to get confidence intervals for this random variable based on the sample $m = f(x) = f(x_1),f(x_2),..,f(x_n).$