# Reconciling optimisation for log-likelihood and Brier score

Both log-likelihood and Brier score are proper scoring rules. As such, they reach the optimum when the predicted probabilities match the true ones. Since there is only one true probability for each predictor vector ($$\textbf{x}$$), minimising either of the two (or, for that matter, any other proper scoring rule) should lead to the true model - assuming the model is correctly specified, right? And if the model is not correctly specified, no scoring rule can lead to the true model, anyway.

So, if my reasoning above is correct, the optima for log-likelihood and Brier score should coincide, despite them having completely different forms. For example, log-likelihood can go into infinity, while Brier score plateaus. If they don't coincide, it indicates that the model was not correct. Is there anything to learn about the mismatch between the model and the data from analysing the differences between the models obtained by optimising for different scores?

But, isn't optimising Brier score simply least squares on a function bound to $$[0, 1]$$? Shouldn't we avoid least squares for probabilities, due to heteroscedasticity (and non-normality of the errors)? What am I missing?

• Here is a comparison between the log and the Brier scores that may be helpful: Why is LogLoss preferred over other proper scoring rules? Commented Mar 1 at 13:01
• Your reasoning is correct -- in the limit, optimizing any proper scoring rule leads to the correct model, assuming it's in your class.
– usul
Commented Mar 2 at 2:13
• "If they [the minima of the scores] don't coincide, it indicates that the model was not correct." Remember that proper scoring rules are minimized in expectation by the true probabilities, and that empirically calculated scores are just estimates of the expectations. So different minima may just be due to sampling variability. Finite-sample theory for proper scoring rules is not well developed. (Plus the question of course remains whether your instances all do have the same class membership probability in the first place.) Commented Mar 2 at 17:37