Assume that we have $n$ training data $x_1, ... x_n$ , generated by a probability model $P(x;\theta)$. We want to estimate the parameters $\theta$ of the model based on the observations.

In non-Bayesian statistics, we can estimate the parameters by a maximum likelihood function. To calculate the MLE, optimization methods could be used (like stochastic gradient decent, EM), and we can find a solution after convergence.

To compare, Bayesian way is to estimate the parameters by sampling from a posterior distribution (like Gibbs sampling, slice sampling), and after the MC chain running to a steady state, we can get samples from the posterior distribution and using some estimators to estimate the parameters.

Is there any relations between between these two parameter-estimating techniques?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.