# Bootstrapping for neural network validation

I need to validate a specific/trained neural network for classification, and I'm planning to use bootstrapping for this purpose. My idea is to keep fixed the trained network and generate bootstrap samples only for the test set, and then obtain statistics about accuracy/false pos/false neg for that fixed network.

Do you think this approach is adequate? I'm asking because, as far as I've seen, the typical approach is to train the network at each bootstrap sample (after holding out the test samples). Instead, I want to obtain statistics about classification accuracy for a specific network.

Another question: in my case, the samples are obtained uniformly from $\mathbb{R}^n$, unlike the typical case when you resample from a finite set of observations. Then, is it still correct to speak of "bootstrapping"?

Thank you!

Bootstrap is a statistical tool help us emulate the process of acquiring new sample set, usually when it is impossible or inconvenient to do so. It resamples from an existing sample set: If you are resampling from the training set to generate the testing sets, it will not be adequate since all these testing cases have been used during your training; if you are resampling from $\mathbb{R}^n$, this is randomly, independently generating test set from the population, which is adequate, but cannot be called bootstrap.
• Many thanks for clarifying the meaning of bootstrap. What do you think of the validation scheme I described (i.e., resample only the test set from $\mathbb{R}^n$ and keep the network fixed)? – Nicola Paoletti Oct 12 '17 at 20:44
• It looks good. But a more standard way is to obtain data needed for your model(training/testing) in the first place. For example obtain 10,000 samples. Then divided them into training and testing data. (7,000 for training and 3,000 for testing). From my understanding, what you are doing is training the model used the obtained 7,000 samples and then sampling another 3,000 data from $\mathbb{R}^n$ for testing. This is a little bit odd, but will not cause any problem for validating your model. – X. Zhang Oct 12 '17 at 20:56
• Yes, just to confirm, the workflow would be: 1. Train the model using samples drawn from $\mathbb{R}^n$; 2. Sample N times (e.g. N=1000) the testing set from $\mathbb{R}^n$; 3. Obtain statistics (e.g. mean accuracy and S.E.) by validating the trained model with all test datasets – Nicola Paoletti Oct 12 '17 at 21:57