i have split my training data 30/70 and trained models, my models are performing really well on the training set but i have a large unlabelled dataset where i want to do inference, how could i measure accuracy on this dataset for which i don't have the correct answer.
I want to know how well my model does in production. how would you validate a large dataset say 600k examples you are doing inference on, i have around 40 classes which i am predicting. If i validate a sample of this by hand can i then extrapolate those metrics to the whole population. Have you come across similar examples ? My models are performing really well on the training data 98% F1 scores. I was thinking of using a bayesian estimation to find out how well my does in production. I don't have feedback loop so i need to check the predictions manually.