# Machine learning model metrics vs predicted probability?

If I train a machine learning model, how do you explain the relation between model metrics and a predicted output?

For example with a Logistic Regression, through a series of parameter tuning CV iterations I find a model that preforms 0.95 overall accuracy, 0.75 precision on positive class, and 0.66 recall on positive class. Now when I make some predictions on two new datapoints, I get 0.45 and 0.99 as the predicted probabilities. How would you explain these outputs to someone? Would you say the 0.45 is 45% confident/likely to be a positive class, while the model is still only 0.75 precise?

My business goal is to make a model that can say "we are X% sure that something will happen". Which numbers are valid to use to define X?

The output given you by logistic regression are probabilities, more precisely their point estimates. Since we are dealing with random variables, the estimates are uncertain and we have a number of metrics that quantify this uncertainty. Standard errors tell us how uncertain are we with our estimates, while metrics such as accuracy, preferably measured on a separate test set, tell us how uncertain are we about future predictions that would be made using your model. It is the same as if you measured the temperature using a thermometer, it says that the temperature is X, but the measurement error is $\pm$Y. Usually non-statisticians want point estimates and don't want to hear about the uncertainty, while statisticians prefer to report all this to make their reports more precise.