For example, we have log-likelihood function $\ell(\theta)$ and maximum likelihood estimator $\hat \theta_{ML}$ obtained as $\ell(\theta)' = 0$. What would be $$Cov(\ell(\theta), \hat \theta_{ML}) ?$$

For simplicity let's assume that MLE is unbiased.

  • $\begingroup$ should it be $\hat\theta_{ML}$ in the argument of $\ell$? $\endgroup$ – innisfree Mar 15 at 7:04
  • $\begingroup$ Well, I think this is a function of the parameter $\theta$ $\endgroup$ – student Mar 15 at 7:06
  • $\begingroup$ What are you viewing as random here? The data, the parameter, or both? $\endgroup$ – whuber Mar 15 at 14:39
  • $\begingroup$ Data. Parameters are usually assumed to be fixed. $\endgroup$ – student Mar 15 at 14:46
  • 1
    $\begingroup$ In a Bayesian setting parameters are treated as random. When the randomness occurs only through the data, I believe your question has no general answer: it must depend on the model as well as $\theta$ itself. In many cases this covariance must be tiny, but in some circumstances--such as when $\theta$ lies near the boundary of the parameter space--it cannot vanish. $\endgroup$ – whuber Mar 16 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.