How do you calculate standard errors for a transformation of the MLE?

Question

I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance property of the MLE directly gives me a point estimate for $p$, but I am not sure how to compute s.e for $p$. Thank in advance for any suggestion or reference.

Answer 1

The Delta method is used for this purpose. Under some standard regularity assumptions, we know the MLE, $\hat{\theta}$ for $\theta$ is approximately (i.e. asymptotically) distributed as

$$ \hat{\theta} \sim N(\theta, \mathcal{I}^{-1}(\theta)) $$

where $\mathcal{I}^{-1}(\theta)$ is the inverse of the Fisher information for the entire sample, evaluated at $\theta$ and $N(\mu,\sigma^{2})$ denotes the normal distribution with mean $\mu$ and variance $\sigma^{2}$. The functional invariance of the MLE says that the MLE of $g(\theta)$, where $g$ is some known function, is $g(\hat{\theta})$ (as you pointed out) and has approximate distribution

$$ g(\hat{\theta}) \sim N( g(\theta), \mathcal{I}^{-1}(\theta) [g'(\theta)]^{2} ) $$

where you can plug in consistent estimators for the unknown quantities (i.e. plug in $\hat{\theta}$ where $\theta$ appears in the variance). I would assume the standard errors you have are based on the Fisher information (since you have MLEs). Denote that standard error by $s$. Then the standard error of $e^{\hat{\theta} }$, as in your example, is

$$ \sqrt{s^{2}e^{2 \hat{\theta}}} $$

I may be interpreting you backwards and in reality you have the variance of the MLE of $\theta$ and want the variance of the MLE of $\log(\theta)$ in which case the standard would be

$$ \sqrt{ s^{2}/\hat{\theta}^{2} } $$

Answer 2

Macro gave the correct answer on how to transform standard errors via the delta method. Though the OP specifically asked for the standard errors, I suspect that the objective is to produce confidence intervals for $p$. Besides computing estimated standard errors of $\hat{p}$ you can directly transform a confidence interval, $[q_1, q_2]$, in the $q$-parametrization to a confidence interval $[\exp(q_1), \exp(q_2)]$ in the $p$-parametrization. This is perfectly valid, and it may even be a better idea depending on how well the normal approximation used to justify a confidence interval based on standard errors works in the $q$-parametrization versus the $p$-parametrization. Moreover, the directly transformed confidence interval will fulfill the positivity constraint.