Markov chain Monte Carlo (MCMC) for Maximum Likelihood Estimation (MLE) I am reading a 1991 conference paper by Geyer which is linked below. In it he seems to elude to a method that can use MCMC for MLE parameter estimation
This excites me since, I have coded BFGS algorithms, GAs and all sorts of these horrible hand wavy lucky-dip methods of finding global minima necessary to extract the estimation of parameters from MLEs.
The reason it excites me is that if we can guarantee convergence of the MCMC to a fixed point (e.g. a sufficient criterion would be satisfying detailed balance) then we can obtain parameters without minimising an MLE.
Conclusion is, therefore, that this provides a generic method to obtain the global minima, modulo constraints imposed above and in the paper. There are a number of algorithms for MCMC e.g. HMC that are well mapped for high dimensional MCMC problems and I would assume that they would outperform traditional gradient descent methods.
Question


*

*Am I correct that this paper provides a theoretical basis for the usage of MCMC to obtain parameter estimates from MLEs?

*Can one use an MCMC algorithm in certain circumstances, as outlined in the paper, to extract parameters from the MLE bypassing the needs for methods like Genetic Algorithms and BFGS etc.
Paper
Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. Computing Science and Statistics: Proc. 23rd Symp. Interface, 156–163.

Abstract
Markov chain Monte Carlo (e.g., the Metropolis algorithm and Gibbs sampler) is a general tool for simulation of complex stochastic processes useful in many types of statistical inference. The basics of Markov chain Monte Carlo are reviewed, including choise of algorithms and variance estimation, and some new methods are introduced. The use of Markov chain Monte Carlo for maximum likelihood estimation is explained and its performance is compared with maximum pseudo likelihood estimation.

Note: Sections 1-6 are boring and you probably know them already if you got this far. In Section 7 he gets to the interesting but of what he terms “Monte Carlo Maximum Likelihood”
More resources
control+f for “Geyer”


*

*http://www.stats.ox.ac.uk/~snijders/siena/Mcpstar.pdf

*http://ecovision.mit.edu/~sai/12S990/besag.pdf (section 2.4)

 A: If I understand correctly, you are excited about MCMC in the case of multi-modal target functions. Your reasoning is that MCMC methods search the global parameter space, rather than just shooting up the closest mode and stopping. 
While theoretically true, in practice, MCMC often behaves somewhat similarly to hill climbing methods: once they find a local mode, they often stay around that mode. Unlike hill climbing methods, there is a positive probability that they will leave the mode, so theoretically it will explore the global space if let run long enough. However, for most samplers, this probability is so extremely small that is unreasonable to run the chain long enough to have any assurance that the sampler will properly explore the global space. 
Of course, there are samplers that try to remedy this by attempting to take occasional outlier steps (i.e. see if it can escape the local mode). But I don't think these samplers are going to be at all competitive, in regards to optimization, with standard optimization methods for exploring multi-modal surfaces (i.e. particle swarm, etc.). 
A: MCMC doesn't converge to a fixed point generally. Convergence is to the stationary distribution of a Markov chain. The draws are different, but, loosely, the distribution they are drawn from becomes fixed. 
MCMC methods generally suffer from similar issues to other optimisation methods. E.g. it is easy to design chains which rarely escape from local minima. There is a whole literature of tricks to solve such problems for various models.
That said and in response to your second question, here's a quick and dirty way MCMC can be used for parameter estimation:


*

*Run the chain, generating parameter samples.

*Get the likelihood under each sample of the parameters. 

*Compare the MCMC samples' likelihoods to your favourite MLE.

*If any of the MCMC samples do better, it wasn't really a global MLE.

