How to understand maximum likelihood estimation from an objective Bayesian paradigm?

Question

I am trying to understand maximum likelihood estimation from an objective Bayesian/Jaynesian paradigm. My current understanding is that:

There is a parametric family of functions f(x; theta) indexed by a tensor theta.

And there is a mapping from the set of tensors to the set of propositions model(theta) that maps a tensor theta to a proposition which we refer to as a “model”.

The probability(X = x | model(theta)), i.e. probability that X takes on the value x given that the proposition model(theta) is true, is equal to f(x; theta) for any tensor theta.

The dataset is a set of propositions: X = x_1, X = x_2, ..., X = x_n.

probability(model(theta) | dataset) is proportional to probability(dataset | model(theta)) * probability(model(theta)).

Assuming a flat prior, i.e. probability(model(theta)) = alpha for any tensor theta, we can ignore the probability(model(theta)) term.

probability(dataset | model(theta)) = probability(X = x_1 | model(theta)) * ... * probability(X = x_n | model(theta)) = f(x_1; theta) * ... * f(x_n; theta)

The goal of maximum likelihood estimation is to find the value of theta which maximises the probability(model(theta) | dataset), i.e. to find the most plausible “model” in the range of model(theta) given the dataset, by maximising the above expression.

Is this a correct understanding of maximum likelihood estimation from a Jaynesian viewpoint?

Thank you

Answer 1

Maximum likelihood estimation does not really fit within an objective Bayes perspective for at least two reasons:

1. the Bayesian paradigm is concerned with full inference rather than point estimation and producing the posterior distribution is its primary goal. Point estimates require a decision theoretic addendum, through a loss function, and maximum likelihood estimators are not universal Bayes estimators, as there is no loss function that always return the maximum likelihood estimator, even though they may coincide for some specific distributions (like the Normal mean under quadratic loss);

2. the maximum likelihood estimator is invariant by reparameterisation, while the maximum a posteriori is not. Even though some parameterisation see MAP (which is arguably un-Bayesian) and MLE coincide, assuming the associated prior is flat, most changes of parameterisation see the coincidence vanish. (In simpler terms, the flat or uniform prior does not remain flat or uniform by a change of variable. This was an early criticism of this choice of prior by Chrystal, Boole, Lhostes, &tc.)

There exists an earlier X validated entry on the links between MaxEnt à la Jaynes and maximum likelihood.