Suppose an estimator $\hat\theta_T$ is defined as the value of $\theta$ maximizing: $$\sum_{t=1}^T{l(y_t|\theta)}+\mu_T g(\theta),$$ where $l(y_t|\theta)$ is the log-likelihood of observation $t$, $\mu_T$ determines the strength of penalization (possibly a function of the sample size $T$), and $g(\theta)$ is a smooth penalty.

The Maximum a Posteriori (MAP) case corresponds to $\mu_T=1$ with $g$ the log-prior.

This is biased in finite samples in general (with bias potentially O(1)), and will be inconsistent if e.g. $\mu_T=O(T)$.

Are there techniques for bias correcting such an estimator, e.g. using information about the derivatives of $g$?

  • $\begingroup$ Of course. But MLE bias is O(1/T). This bias is potentially O(1). $\endgroup$ – cfp Jan 22 '20 at 14:17
  • $\begingroup$ If $\mu_T=O(T)$ this is no longer Bayesian. $\endgroup$ – Xi'an Jan 22 '20 at 14:17
  • $\begingroup$ I said that the MAP case was when $\mu_T=1$. $\endgroup$ – cfp Jan 22 '20 at 14:22
  • $\begingroup$ Needs to be general to be useful I'm afraid. $\endgroup$ – cfp Jan 25 '20 at 21:01
  • $\begingroup$ Bias correcting ML is standard. It just needs 3rd derivatives of the likelihood. See Wikipedia. But I'm happy if your answer results in an estimator that is only as biased as ML, i.e. the resulting bias is O(1/T). $\endgroup$ – cfp Jan 25 '20 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.