I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance property of the MLE directly gives me a point estimate for $p$, but I am not sure how to compute s.e for $p$. Thank in advance for any suggestion or reference.

  • $\begingroup$ Can't you use the same MLE routine to compute a point estimate and s.e. for $p$ directly? $\endgroup$
    – whuber
    Jul 5, 2012 at 2:23

2 Answers 2


The Delta method is used for this purpose. Under some standard regularity assumptions, we know the MLE, $\hat{\theta}$ for $\theta$ is approximately (i.e. asymptotically) distributed as

$$ \hat{\theta} \sim N(\theta, \mathcal{I}^{-1}(\theta)) $$

where $\mathcal{I}^{-1}(\theta)$ is the inverse of the Fisher information for the entire sample, evaluated at $\theta$ and $N(\mu,\sigma^{2})$ denotes the normal distribution with mean $\mu$ and variance $\sigma^{2}$. The functional invariance of the MLE says that the MLE of $g(\theta)$, where $g$ is some known function, is $g(\hat{\theta})$ (as you pointed out) and has approximate distribution

$$ g(\hat{\theta}) \sim N( g(\theta), \mathcal{I}^{-1}(\theta) [g'(\theta)]^{2} ) $$

where you can plug in consistent estimators for the unknown quantities (i.e. plug in $\hat{\theta}$ where $\theta$ appears in the variance). I would assume the standard errors you have are based on the Fisher information (since you have MLEs). Denote that standard error by $s$. Then the standard error of $e^{\hat{\theta} }$, as in your example, is

$$ \sqrt{s^{2}e^{2 \hat{\theta}}} $$

I may be interpreting you backwards and in reality you have the variance of the MLE of $\theta$ and want the variance of the MLE of $\log(\theta)$ in which case the standard would be

$$ \sqrt{ s^{2}/\hat{\theta}^{2} } $$

  • 1
    $\begingroup$ Just a side note: there are also appropriate multivariate extensions whereby the derivatives are replaced by gradients, and the multiplications have to be matrix multiplications, so there's a bit more headache in figuring out where the transpose goes. $\endgroup$
    – StasK
    Aug 18, 2011 at 15:52
  • 1
    $\begingroup$ Thanks for pointing that out StasK. I believe in the multivariate case the asymptotic covariance of $g(\hat{\theta})$ is $\nabla g(\theta)' \mathcal{I}(\theta)^{-1} \nabla g(\theta)$ $\endgroup$
    – Macro
    Aug 18, 2011 at 15:55
  • $\begingroup$ (+1) I added a link to the regularity assumptions (and some other things) since it isn't clear whether these are satisfied in the OP's problem. I might have said that $\hat{\theta}$ is asymptotically normal and not approximately normal, since the convergence rates can be slow at times. $\endgroup$
    – MånsT
    Jul 5, 2012 at 7:08
  • $\begingroup$ Thank you @MånsT, I also did clarify that I meant asymptotically when I said approximately :) $\endgroup$
    – Macro
    Jul 5, 2012 at 11:48

Macro gave the correct answer on how to transform standard errors via the delta method. Though the OP specifically asked for the standard errors, I suspect that the objective is to produce confidence intervals for $p$. Besides computing estimated standard errors of $\hat{p}$ you can directly transform a confidence interval, $[q_1, q_2]$, in the $q$-parametrization to a confidence interval $[\exp(q_1), \exp(q_2)]$ in the $p$-parametrization. This is perfectly valid, and it may even be a better idea depending on how well the normal approximation used to justify a confidence interval based on standard errors works in the $q$-parametrization versus the $p$-parametrization. Moreover, the directly transformed confidence interval will fulfill the positivity constraint.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.