1
$\begingroup$

Why do you have to define a likelihood function then say you are maximising the likelihood function? Why can't you just maximize $p(x|\theta)$, which is an expression in $\theta$ ? What is technically wrong with doing that?

$\endgroup$
  • $\begingroup$ Because a likelihood is actually not a conditional probability: theta is a parameter, not a random variable. $\endgroup$ – Vincent Guillemot Oct 16 '17 at 11:48
  • 1
    $\begingroup$ I'm sorry I've looked at the linked questions and I still cannot understand why you can't maximize p(x|θ), I feel that they are somewhat different questions and I am not sure which of the answers answer my question, I feel that my question is not really what is the difference between likelihood and probability, nor whether likelihood is a pdf (it doesn't even sum to 1)? $\endgroup$ – liyuan Oct 16 '17 at 12:22
  • 1
    $\begingroup$ @liyuan both linked threads (especially second one) answer your question. You can't maximize $p(x|\theta)$ because $\theta$ is not a random variable, so you cannot condition on it and such conditional probability does not exist. You could maximize it only if you assumes $\theta$ to be a random variable, but then you would need to assume a probability distribution (prior) for it and use Bayesian approach rather then MLE. $\endgroup$ – Tim Oct 16 '17 at 12:25
  • $\begingroup$ Have you looked at this answer: stats.stackexchange.com/a/224623/2116? Likelihood function treats $p(x|\theta)$ as a function of $\theta$, with $x$ fixed, whereas probability densitye function treats $p(x|\theta)$ as a function of $x$ with $\theta$ fixed. Hence maximisation in both cases gives a different result. $\endgroup$ – mpiktas Oct 16 '17 at 12:33
  • $\begingroup$ Can you maximize p(x;θ) = f(θ)? $\endgroup$ – liyuan Oct 16 '17 at 12:34