Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance property of the MLE directly gives me a point estimate for $p$, but I am not sure how to compute s.e for $p$. Thank in advance for any suggestion or reference.

share|improve this question
Can't you use the same MLE routine to compute a point estimate and s.e. for $p$ directly? – whuber Jul 5 '12 at 2:23

The Delta method is used for this purpose. Under some standard regularity assumptions, we know the MLE, $\hat{\theta}$ for $\theta$ is approximately (i.e. asymptotically) distributed as

$$ \hat{\theta} \sim N(\theta, \mathcal{I}^{-1}(\theta)) $$

where $\mathcal{I}^{-1}(\theta)$ is the inverse of the Fisher information for the entire sample, evaluated at $\theta$ and $N(\mu,\sigma^{2})$ denotes the normal distribution with mean $\mu$ and variance $\sigma^{2}$. The functional invariance of the MLE says that the MLE of $g(\theta)$, where $g$ is some known function, is $g(\hat{\theta})$ (as you pointed out) and has approximate distribution

$$ g(\hat{\theta}) \sim N( g(\theta), \mathcal{I}^{-1}(\theta) [g'(\theta)]^{2} ) $$

where you can plug in consistent estimators for the unknown quantities (i.e. plug in $\hat{\theta}$ where $\theta$ appears in the variance). I would assume the standard errors you have are based on the Fisher information (since you have MLEs). Denote that standard error by $s$. Then the standard error of $e^{\hat{\theta} }$, as in your example, is

$$ \sqrt{s^{2}e^{2 \hat{\theta}}} $$

I may be interpreting you backwards and in reality you have the variance of the MLE of $\theta$ and want the variance of the MLE of $\log(\theta)$ in which case the standard would be

$$ \sqrt{ s^{2}/\hat{\theta}^{2} } $$

share|improve this answer
Just a side note: there are also appropriate multivariate extensions whereby the derivatives are replaced by gradients, and the multiplications have to be matrix multiplications, so there's a bit more headache in figuring out where the transpose goes. – StasK Aug 18 '11 at 15:52
Thanks for pointing that out StasK. I believe in the multivariate case the asymptotic covariance of $g(\hat{\theta})$ is $\nabla g(\theta)' \mathcal{I}(\theta)^{-1} \nabla g(\theta)$ – Macro Aug 18 '11 at 15:55
(+1) I added a link to the regularity assumptions (and some other things) since it isn't clear whether these are satisfied in the OP's problem. I might have said that $\hat{\theta}$ is asymptotically normal and not approximately normal, since the convergence rates can be slow at times. – MånsT Jul 5 '12 at 7:08
Thank you @MånsT, I also did clarify that I meant asymptotically when I said approximately :) – Macro Jul 5 '12 at 11:48

Macro gave the correct answer on how to transform standard errors via the delta method. Though the OP specifically asked for the standard errors, I suspect that the objective is to produce confidence intervals for $p$. Besides computing estimated standard errors of $\hat{p}$ you can directly transform a confidence interval, $[q_1, q_2]$, in the $q$-parametrization to a confidence interval $[\exp(q_1), \exp(q_2)]$ in the $p$-parametrization. This is perfectly valid, and it may even be a better idea depending on how well the normal approximation used to justify a confidence interval based on standard errors works in the $q$-parametrization versus the $p$-parametrization. Moreover, the directly transformed confidence interval will fulfill the positivity constraint.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.