A few more remarks in addition to the points made by Richard Price:
- The MLE always corresponds to the uniform prior (the MAP of the
uniform prior is the MLE).
This is incorrect for a simple if often overlooked reason: the MLE does not require a dominating measure on the parameter space while a Bayesian approach does. This means that both "the" flat (constant) prior and "the" MAP are actually depending on the choice of the dominating measure. An alternative explanation (already made in a comment) is that the MLE is invariant by reparameterisation, that is under any bijective transform of the parameter, while a flat prior does not remain constant under bijective transforms and the MAP is not invariant by reparameterisation. My general view on MAPs is that they are not Bayesian procedures.
- Sometimes a uniform prior is not possible (when the data lacks an
upper or lower bound).
This is both correct and incorrect. Choosing a Uniform prior $\mathcal U(a,b)$ is always possible, but requires a choice of $a$ and $b$. If the prior density is constant over the entire parameter space (against the chosen dominating measure) then it is not a Uniform density because it is not a probability density. The prior then becomes improper, that is, a $\sigma$-finite measure.
- Non-Bayesian analysis, which uses the MLE instead of the MAP,
essentially sidesteps or ignores the issue of modeling prior
information and thus always assumes that there is none.
This is too vague a statement to validate or invalidate. As signaled by Richard Price]1, the choice of a model is a type of information, which may become increasingly Bayesian when bringing in random effects for instance. Further, non-Bayesian analysis is not defined as an approach per se.
- Non-informative (also called reference) priors correspond to the
maximizing the Kullback-Leibler divergence between posterior and
prior, or equivalently the mutual information between the parameter 𝜃
and the random variable 𝑋.
Correct: Reference priors in the specific sense of Bernardo (1979), Berger, Bernardo, and Sun (2009), and others, are maximising the expected Kullback-Leibler divergence between prior and posterior for the parameter(s) of interest. As this is usually impossible when considering proper priors, it gets complicated.
- Sometimes the reference prior is not uniform, it can also be a
Jeffreys prior instead.
This is again a vague and not so useful statement. For the same reason as above, namely the lack of invariance under reparameterisation, the reference prior [assuming a specific definition of said prior] is almost never uniform. Jeffreys' approach enjoys invariance under reparameterisation in the sense that its definition is consistent under changes of parameterisation.
- Bayesian inference always uses the MAP and non-Bayesian inference
always uses the MLE.
This is incorrect, for both parts. Bayesian inference always uses the full posterior distribution and only derives procedures like point estimates in cases a decision is required and a loss function provided. MAP estimates are not available as decision-theoretic procedures. Non-Bayesian inference covers all possible answers to an inference problem and hence cannot be characterised.