When is metropolis Hastings algorithm useful

Question

I need to do a data analysis project and am considering the Metropolis Hastings algorithm to estimate the parameters of a logistic regression model. I would draw from the complete data log likelihood and use a multivariate normal proposal distribution that has dimension equal to the number of parameters. Does this make sense? And it doesn’t seem like there is too much of a point to do this because we can directly get the coefficient estimates from logistic regression. Is this right; if so, is there any benefit to running metropolis Hastings on it? And if not, is there an example of parameter estimation that metropolis Hastings would work but a glm or other direct method wouldn’t? Thank you.

Answer 1

Metropolis-Hastings is one of many mcmc algorithms. Those algorithms are designed for sampling from arbitrary probability distributions. If you just want to obtain frequentist point estimates for the parameters of logistic regression, then you don't need MCMC sampling. Moreover, you won't be able to use it for this purpose, since estimating the parameters is an optimization problem and MCMC aren't optimization algorithms. Metropolis-Hastings algorithm does not do the estimation.

You would use MCMC when you treat logistic regression as a Bayesian model, i.e. assume priors for the parameters and are interested in learning the posterior distribution of the parameters. In such a case, MCMC is used to sample from the posterior distribution to obtain a numerical approximation of it.