In Andrew Ng's Coursera class on Machine Learning, we learned to use a Gaussian distribution
to detect anomalous examples when $p(x)<\epsilon$ where $x_j$ are independent features of some test example, and $\mu_j$ and $\sigma^2_j$ are the mean and standard deviation from a training set consisting of non-anomalous examples, and $\epsilon$ is a threshold chosen from a cross-validation set.
I have a data set where I know all of the anomalous examples. I randomly assigned 60% of the good examples to the training set to calculate $\mu_j$ and $\sigma^2_j$ and 40% of the good examples to a cross-validation set to calculate the value of $\epsilon$ that minimizes the $F$ Score. All of the bad examples went into the cross-validation set. I was able to determine the particular features $x_j$ to use based on my knowledge of the problem and the results of the cross-validation set.
Now, I need to provide $\mu_j$ and $\sigma^2_j$ and $\epsilon$ as official values for screening all incoming parts in the manufacturing process at our company.
The problem is there is some variability in $\epsilon$ (and the other statistics) based on how I randomly divide between the training and cross-validation set. I have read about K-fold cross-validation and bootstrap cross-validation.
If I use either of these methods to generate a set of training sets and cross-validation sets, what parameters $\mu_j$ and $\sigma^2_j$ and $\epsilon$ should I report as the official numbers? The average over all partitions?
Should I even generate a set of training sets and cross-validation sets? Should I just pick one with the highest $F$ score and be done?