How to tackle a problem where dataset has high variance involved Suppose, If I have twenty confidence scores for twenty different images as defined below object classified as cats, and its average is 58.9%. For first image, the confidence score is 25.9% and for the second image it's 93.3% and for the third image it's 92.8% and so on. So there is a high variance involved in these data.
cat [[0.2596674 ]]
cat [[0.9336238 ]]
cat [[0.9280105 ]]
cat [[0.40792587]]
cat [[0.29937932]]
cat [[0.7761701]]

If I have to decide between two different systems for classification, system-I and system-II, based on the average score, I retrieved 58.9% for system-II (Neural Network model), I am deciding to go for system-I (Neural Network model) for classification, and provide my input image to it, but I am deciding based on average which is wrong since high variance is involved, maybe for the first-time system-I will provide 76.5% and on next time it may provide a poor performance 23.1%.
How to tackle a problem like this help is highly appreciated. Thanks
 A: Note: still under construction
interpretation:
I think you might be asking about the bias-variance tradeoff.  How do you minimize the miss-rate?
tool:
I like to use 'R' to simulate answers to questions like this.
slight rephrase:
Given two classification schemes, one that has a higher mean accuracy, but higher variance, and another that has lower mean accuracy but lower variance, how do I choose the scheme that is more likely to work well?
Engaging it:
There is hand-waving to be had, but I think the normal distribution is not a terrible 2-parameter starting point for an exploration like this.
There are 3 variables that we are going to want to "sweep" and not 4.  I could imagine sweeping two means and two variances if this were not a bounded domain, but because it is accuracy and not the real line, I think we have to look at it differently.  I think we can pin one mean and variance, and consider the other in reference to that.  So I like the reference having 50/50 and sweeping the other in terms of that.  I think we could do a set of fixed values for the variance  instead of "every value" and look at a smaller comparison set.
Note: still under construction
