Consider i.i.d observation vector ${\bf x}$ from a distribution $F$ depending on vector of parameters $\boldsymbol{\theta}$ and single parameter $\alpha$.

We would like to estimate parameters [$\boldsymbol{\theta}, \alpha$] using maximum likelihood estimation but we also know that $\alpha$ is a realisation from a $N(\mu, \sigma^2)$.

If we didn't know the distribution of $\alpha$ we could just maximise the objective function $f({\bf x} | \boldsymbol{\theta}, \alpha)$ but knowing the distribution for $\alpha$, what does the objective function to maximise become?

  • $\begingroup$ In what sense are you going to "estimate" the random variable $\alpha$, by maximizing some objective function that includes it? $\endgroup$ – Alecos Papadopoulos Apr 18 '14 at 15:50
  • $\begingroup$ Yes, exactly. I was thinking something like $f({\bf x} | \boldsymbol{\theta}, \alpha) * \phi(\alpha)$ but this is only what my intuition is saying. $\endgroup$ – rwolst Apr 18 '14 at 16:33
  • 2
    $\begingroup$ The issue is not the form of the function. When parameters are treated as random variables, you are in Bayesian estimation framework, so no objective function and no maximization approach. $\endgroup$ – Alecos Papadopoulos Apr 18 '14 at 16:44
  • $\begingroup$ Isn't that the definition for MAP estimation? $\endgroup$ – Royi Apr 18 '14 at 16:53
  • 1
    $\begingroup$ Right, it does seem to be MAP estimation, in which case $f({\bf x} | \boldsymbol{\theta}, \alpha) * \phi(\alpha)$ is indeed the objective function to maximise and the non-random parameters can be thought of as coming from a probability distribution with all the probability at one point. $\endgroup$ – rwolst Apr 18 '14 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.