# Observed Fisher information under a transformation

From "In All Likelihood: Statistical Modeling and Inference Using Likelihood" by Y. Pawitan, the likelihood of a re-parameterization $\theta\mapsto g(\theta)=\psi$ is defined as $$L^*(\psi)=\max_{\{\theta:g(\theta)=\psi\}} L(\theta)$$ so that if $g$ is one-to-one, then $L^*(\psi)=L(g^{-1}(\psi))$ (p. 45). I am trying to show Exercise 2.20 which states that if $\theta$ is scalar (and I presume that $g$ is supposed to be a scalar function as well), then $$I^*(g(\hat{\theta}))=I(\hat{\theta})\left|\frac{\partial g(\hat{\theta})}{\partial \hat{\theta}}\right|^{-2},$$ where $$I(\theta)=-\frac{\partial^2}{\partial\theta^2} l(\theta)$$ is the observed Fisher information and $l(\theta)=\log L(\theta)$.

If $g$ is one-to-one then this is straightforward using the chain-rule and the invariance principle. I am just wondering about a few things:

1. Why does he insist on writing the absolute value? This could be left out, right?
2. By $\frac{\partial g(\hat{\theta})}{\partial \hat{\theta}}$ he means the function $\frac{\partial g(\theta)}{\partial \theta}$ evaluated at $\theta=\hat{\theta}$, right? If this is the case, then isn't it a poor choice of notation? I believe that the usual shorthand notation for this woruld be $\frac{\partial g(\hat{\theta})}{\partial \theta}$.
3. How is this shown when $g$ is not necessarily one-to-one?
• The absolute value may stem from the formula that is used to transform a probability density. But for the score you, indeed, do not use the absolute values. It is not only superfluous, it is also wrong (it only doesn't matter because there is a square in this way of expressing the Fisher information). Nov 14 '20 at 14:55

2. You're right. An even better notation would be $$\frac{dg(\theta)}{d\theta}\Bigg|_{\theta=\hat{\theta}}$$.
3. It doesn't hold in general. Fix some $$\psi_0$$ and define $$g:\mathbb{R}\to\mathbb{R}$$ by $$g(\theta)=\psi_0$$. The rhs would be undefined since the derivative is zero for every $$\theta$$.
For smooth one-to-one $$g$$ with $$\psi=g(\theta)$$. Since, $$d/d\psi = d\theta/d\psi\cdot d/d\theta$$, we have \begin{align} I^*(\psi) &= -\frac{d^2L^*(\psi)}{d\psi^2} = -\frac{d}{d\psi}\left(\frac{dL^*(\psi)}{d\psi} \right) = -\frac{d}{d\psi}\left(\frac{dL^*(\psi)}{d\theta} \frac{d\theta}{d\psi}\right) \\ &= - \frac{d^2L^*(\psi)}{d\theta^2}\left(\frac{d\theta}{d\psi}\right)^2 - \frac{dL^*(\psi)}{d\theta}\frac{d^2\theta}{d\psi^2} \frac{d\theta}{d\psi}\, . \end{align} Therefore, \begin{align} I^*(g(\hat{\theta})) &= -\frac{d^2L^*(g(\hat{\theta}))}{d\theta^2}\left(\frac{d\theta}{d\psi}\right)^2 - \frac{dL^*(g(\hat{\theta}))}{d\theta}\frac{d^2\theta}{d\psi^2} \frac{d\theta}{d\psi} \\ &= -\frac{d^2L(g^{-1}(g(\hat{\theta})))}{d\theta^2} \left(\frac{dg(\theta)}{d\theta}\Bigg|_{\theta=g^{-1}(g(\hat{\theta}))}\right)^{-2} - \frac{dL(g^{-1}(g(\hat{\theta})))}{d\theta}\frac{d^2\theta}{d\psi^2} \frac{d\theta}{d\psi} \\ &= I(\hat{\theta}) \left(\frac{dg(\theta)}{d\theta}\Bigg|_{\theta=\hat{\theta}}\right)^{-2} \, , \end{align} in which we used $$dL(g^{-1}(g(\hat{\theta})))/d\theta=dL(\hat{\theta})/d\theta=0$$.
• Thank you for addressing all my doubts and for that simple counter-example with constant $g$. Your sketch of the regular case is similar to what I've done, so it's all good. Thanks. Oct 4 '14 at 10:54