I am reading a 1991 conference paper by Geyer which is linked below. In it he seems to elude to a method that can use MCMC for MLE parameter estimation
This excites me since, I have coded BFGS algorithms, GAs and all sorts of these horrible hand wavy lucky-dip methods of finding global minima necessary to extract the estimation of parameters from MLEs.
The reason it excites me is that if we can guarantee convergence of the MCMC to a fixed point (e.g. a sufficient criterion would be satisfying detailed balance) then we can obtain parameters without minimising an MLE.
Conclusion is, therefore, that this provides a generic method to obtain the global minima, modulo constraints imposed above and in the paper. There are a number of algorithms for MCMC e.g. HMC that are well mapped for high dimensional MCMC problems and I would assume that they would outperform traditional gradient descent methods.
Am I correct that this paper provides a theoretical basis for the usage of MCMC to obtain parameter estimates from MLEs?
Can one use an MCMC algorithm in certain circumstances, as outlined in the paper, to extract parameters from the MLE bypassing the needs for methods like Genetic Algorithms and BFGS etc.
Markov chain Monte Carlo (e.g., the Metropolis algorithm and Gibbs sampler) is a general tool for simulation of complex stochastic processes useful in many types of statistical inference. The basics of Markov chain Monte Carlo are reviewed, including choise of algorithms and variance estimation, and some new methods are introduced. The use of Markov chain Monte Carlo for maximum likelihood estimation is explained and its performance is compared with maximum pseudo likelihood estimation.
Note: Sections 1-6 are boring and you probably know them already if you got this far. In Section 7 he gets to the interesting but of what he terms “Monte Carlo Maximum Likelihood”
control+f for “Geyer”