2
$\begingroup$

I need to do a data analysis project and am considering the Metropolis Hastings algorithm to estimate the parameters of a logistic regression model. I would draw from the complete data log likelihood and use a multivariate normal proposal distribution that has dimension equal to the number of parameters. Does this make sense? And it doesn’t seem like there is too much of a point to do this because we can directly get the coefficient estimates from logistic regression. Is this right; if so, is there any benefit to running metropolis Hastings on it? And if not, is there an example of parameter estimation that metropolis Hastings would work but a glm or other direct method wouldn’t? Thank you.

$\endgroup$
1
  • $\begingroup$ MCMC is a simulation technique, not an inferential principle. $\endgroup$
    – Xi'an
    May 20, 2021 at 9:06

1 Answer 1

3
$\begingroup$

Metropolis-Hastings is one of many algorithms. Those algorithms are designed for sampling from arbitrary probability distributions. If you just want to obtain frequentist point estimates for the parameters of logistic regression, then you don't need MCMC sampling. Moreover, you won't be able to use it for this purpose, since estimating the parameters is an optimization problem and MCMC aren't optimization algorithms. Metropolis-Hastings algorithm does not do the estimation.

You would use MCMC when you treat logistic regression as a Bayesian model, i.e. assume priors for the parameters and are interested in learning the posterior distribution of the parameters. In such a case, MCMC is used to sample from the posterior distribution to obtain a numerical approximation of it.

$\endgroup$
2
  • $\begingroup$ Thanks! This and the links are helpful. I’ll try a multivariate normal prior with mean and variance given by a preliminary logistic regression and see how it goes. $\endgroup$
    – Jellyfish
    May 20, 2021 at 8:39
  • $\begingroup$ @Stacker keep in mind that M-H is not the most efficient algorithm for such a task, there are more modern alternatives that are available out-of-the-box in software like mc-stan.org or docs.pymc.io $\endgroup$
    – Tim
    May 20, 2021 at 9:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.