Suppose I have a large set of manually labelled data (e.g. 5000+ instances) with one of two lables, A or B, and I intend to build a ML classifier from this data. Using a proper methodology (e.g. cross validation, dividing data into training, testing, and validation sets etc.), suppose I obtain a classification accuracy (or ROC score) of ~85% on the testing set.
If I were to now take this classifier and let it loose on more instances (10,000+) of the exact same type of data, but this time the data has no labels (ie. ground truth), could I safely assume that 85% of these new instances are correctly labelled with respect to our ground truth training set (the original 5000+ instances)?
In addition, if I were to apply simple data analyses (e.g. t-tests) to these 10,000+ newly labelled instances, do there exist any methods to incorporate the fact that these instances may be only 85% correct? In other words is there a way to include in the error of the actual labelling of the data into the significance calculations for the distribution, counts, patterns etc. of the labels?
And last, does the scenario I'm describing sound like a situation that is better suited for semi-supervised learning methods?