# How do you calculate standard errors for a transformation of the MLE?

I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance property of the MLE directly gives me a point estimate for $p$, but I am not sure how to compute s.e for $p$. Thank in advance for any suggestion or reference.

• Can't you use the same MLE routine to compute a point estimate and s.e. for $p$ directly? – whuber Jul 5 '12 at 2:23

The Delta method is used for this purpose. Under some standard regularity assumptions, we know the MLE, $\hat{\theta}$ for $\theta$ is approximately (i.e. asymptotically) distributed as

$$\hat{\theta} \sim N(\theta, \mathcal{I}^{-1}(\theta))$$

where $\mathcal{I}^{-1}(\theta)$ is the inverse of the Fisher information for the entire sample, evaluated at $\theta$ and $N(\mu,\sigma^{2})$ denotes the normal distribution with mean $\mu$ and variance $\sigma^{2}$. The functional invariance of the MLE says that the MLE of $g(\theta)$, where $g$ is some known function, is $g(\hat{\theta})$ (as you pointed out) and has approximate distribution

$$g(\hat{\theta}) \sim N( g(\theta), \mathcal{I}^{-1}(\theta) [g'(\theta)]^{2} )$$

where you can plug in consistent estimators for the unknown quantities (i.e. plug in $\hat{\theta}$ where $\theta$ appears in the variance). I would assume the standard errors you have are based on the Fisher information (since you have MLEs). Denote that standard error by $s$. Then the standard error of $e^{\hat{\theta} }$, as in your example, is

$$\sqrt{s^{2}e^{2 \hat{\theta}}}$$

I may be interpreting you backwards and in reality you have the variance of the MLE of $\theta$ and want the variance of the MLE of $\log(\theta)$ in which case the standard would be

$$\sqrt{ s^{2}/\hat{\theta}^{2} }$$

• Just a side note: there are also appropriate multivariate extensions whereby the derivatives are replaced by gradients, and the multiplications have to be matrix multiplications, so there's a bit more headache in figuring out where the transpose goes. – StasK Aug 18 '11 at 15:52
• Thanks for pointing that out StasK. I believe in the multivariate case the asymptotic covariance of $g(\hat{\theta})$ is $\nabla g(\theta)' \mathcal{I}(\theta)^{-1} \nabla g(\theta)$ – Macro Aug 18 '11 at 15:55
• (+1) I added a link to the regularity assumptions (and some other things) since it isn't clear whether these are satisfied in the OP's problem. I might have said that $\hat{\theta}$ is asymptotically normal and not approximately normal, since the convergence rates can be slow at times. – MånsT Jul 5 '12 at 7:08
• Thank you @MånsT, I also did clarify that I meant asymptotically when I said approximately :) – Macro Jul 5 '12 at 11:48

Macro gave the correct answer on how to transform standard errors via the delta method. Though the OP specifically asked for the standard errors, I suspect that the objective is to produce confidence intervals for $p$. Besides computing estimated standard errors of $\hat{p}$ you can directly transform a confidence interval, $[q_1, q_2]$, in the $q$-parametrization to a confidence interval $[\exp(q_1), \exp(q_2)]$ in the $p$-parametrization. This is perfectly valid, and it may even be a better idea depending on how well the normal approximation used to justify a confidence interval based on standard errors works in the $q$-parametrization versus the $p$-parametrization. Moreover, the directly transformed confidence interval will fulfill the positivity constraint.