MLE: Standard error of function of parameter Let's assume I'm fitting some arbitrary model via maximum likelihood. For simplicity let's assume I have only one parameter of interest, $\beta$. Let's choose a probit model to illustrate, with log-likelihood function
$$\mathcal{L}=\sum_{i=1}^{n}y_{i}\log(\Phi(\beta x_{i}))+(1-y_{i})\log(1-\Phi(\beta x_{i}))$$
Now, say I wanted to restrict $\beta$ to be positive and for convenience I decided to perform the following transformation to constrain $\beta$
$$\beta=f(\gamma)=\exp(\gamma)$$
So when I construct my likelihood function I am placing $\exp(\gamma)$ in my function and after minimising the negative log-likelihood I obtain $\hat{\gamma}$. I am familiar with how to generate the standard error for $\gamma$. Given that $\beta=f(\gamma)$, how do I get the standard error for $\beta$.
 A: If in reality $\beta>0$ then for large enough $n$ you will have $\hat\beta>0$ and you can use the delta method
$$\mathrm{var}[\hat\beta] = \left(\frac{d\beta}{d\gamma}\right)^2\mathrm{var}[\hat\gamma]= \beta^2\mathrm{var}[\hat\gamma]$$
In this case the transformation hasn't really gotten you much, but it's valid.
The problem comes when an unconstrained estimator would have $\hat\beta\leq 0$.  In fact, I'm going to write $\hat\beta_u$ for the unconstrained estimator and $\hat\beta_c$ for the constrained estimator $e^{\hat\gamma}$, so the problem comes when an unconstrained estimator would have $\hat\beta_u\leq 0$.
In that case the constrained estimator has $\hat\beta_c=0$ and $\hat\gamma =-\infty$.
If the unconstrained problem has $\hat\beta_u\leq 0$ with non-negligible probability then the constrained estimator has $\hat\gamma=-\infty$ with non-negligible probability and $\hat\beta_c=0$ with non-negligible probability. In that case $\hat\gamma$ does not have a finite standard error. While $\hat\beta_c$ does have a finite standard error, it does not have an approximately Normal distribution. The standard error can't straightforwardly be used to construct uncertainty intervals, nor can it be estimated simply from the inverse Fisher information.
On the other hand, if $\hat\beta_u\leq 0$ is just an occasional small-sample issue then there isn't much of a problem and you can just use the delta method when $\hat\gamma$ is finite and fudge somehow when $\hat\gamma=-\infty$.
