I am wondering the differences between the likelihood function in Maximum Likelihood Estimation and the likelihood function in Bayes' Theorem. To me, the likelihood function in Bayes' Theorem depends on the prior probability distribution because values of parameters come from the prior distribution and used in the likelihood function. On the other hand, we do not have any prior probability distribution but we can still do Maximum Likelihood Estimation. It is quite confusing.
$\begingroup$
$\endgroup$
0
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
The priors are separate from the likelihood function in Bayes' theorem; it is the very point of that theorem that you can separate the two (three, if you include the marginal distribution). So there is no difference between the likelihood function in frequentist maximum likelihood and the likelihood function in Bayes.
-
$\begingroup$ So, the likelihood function is not dependent on the prior probability? $\endgroup$– oceanusSep 17, 2020 at 7:48
$\begingroup$
$\endgroup$
They are the same. The likelihood is $p(X|\theta)$ where $X$ is the data and $\theta$ is the parameter to be estimated, this term gives the probability of $X$ given $\theta$, so $p(\theta)$ (the prior) does not get involved.
However the posterior probability,$\,\,p(\theta|X)$, does depend on $p(\theta)$ because $p(\theta|X) = \frac{p(X|\theta) p(\theta)}{p(X)} \propto p(X|\theta) p(\theta)$.
$\begingroup$
$\endgroup$
Regarding the likelihood, there is no difference in the mathematical formula and computation.
Suppose there is one data point $y$ ans we assume $y\sim N(\mu, \sigma^2)$. Then the likelihood is: $$\dfrac{1}{\sigma {\sqrt {2\pi }}} e^{-{\frac {1}{2}}\left({\frac {y-\mu }{\sigma }}\right)^{2}}.$$ If any, there is one theoretical difference. That is the way we treat the parameters $\mu$ and $\sigma$. In the Bayesian framework, the parameters are random (that is why we need priors), whereas in the frequentist inference, they are fixed.
