# Where does the efficient locally unbiased estimator $\tilde\theta(m)=\theta_0+\frac{F^{-1}\nabla p_\theta(m)}{p_\theta(m)}$ come from?

In Equation (9), page 9 of (Demkowicz-Dobrzanski et al. 2020), the authors mention that, given a probability distribution $$p_{\boldsymbol\theta}(m)$$ with hidden parameter $$\boldsymbol\theta$$ and $$m$$ labelling the possible outcomes, if the true parameter $$\boldsymbol\theta$$ is close to some known $$\boldsymbol\theta_0$$, then we have a locally unbiased estimator $$\tilde{\boldsymbol\theta}$$ that saturates the CR bound and is thus optimal at $$\boldsymbol\theta_0$$, written as $$\tilde{\boldsymbol\theta}(m) = \boldsymbol\theta_0 + \frac{1}{p_{\boldsymbol\theta}(m)} F^{-1} \nabla p_{\boldsymbol\theta}(m)\big|_{\boldsymbol\theta=\boldsymbol\theta_0},\tag1$$ where $$F$$ is the Fisher information matrix: $$F = \sum_m \frac{\nabla p_{\boldsymbol\theta}(m)[\nabla p_{\boldsymbol\theta}(m)]^T}{p_{\boldsymbol\theta}(m)}.$$

That this estimator is efficient around the true value I can see because $$\mathbb{E}[(\tilde{\boldsymbol\theta}-\boldsymbol\theta_0)^2] = \sum_m \frac{1}{p(m)}\sum_{ijk} (F^{-1})_{ij}(F^{-1})_{ik} \partial_j p(m)\partial_k p(m) \\ = \sum_i (F^{-1}FF^{-1})_{ii} = \operatorname{tr}(F^{-1}).$$ What I'm not clear about is how (1) is derived in the first place. Sure, once I have it I can verify that it works, but what's a way to get to it from scratch? I thought of trying to find the locally unbiased estimator that minimises the variance at the true value, but that would just give me back the trivial estimator $$m\mapsto \boldsymbol\theta_0$$.

Is this particular structure characterised by its being locally unbiased and efficient? It would seem like that's what is being stated when it's discussed in the paper. But isn't the trivial estimator $$\tilde{\boldsymbol\theta}(m)=\boldsymbol\theta_0$$ already enough for that? it's locally unbiased and has zero variance around the true parameter. So how is this one better exactly?

• Haven't check if $\tilde{\boldsymbol{\theta}}$ is locally unbiased(l.u.) or not, but $\boldsymbol{\theta }_0$ is not l.u. IMO. You can see $\boldsymbol{\theta }_0$ does not depend on $m$ hence a constant. Therefore Eq.(5)(the right hand side one) of your reference suggests it's not l.u. Mar 28 at 8:11

More precisely, we're asking the two conditions, spelled out in Eqs. (4) and (5) in the above linked paper: $$\sum_m p_{\theta_0}(m)\hat\theta(m)=\theta_0, \qquad \sum_m \Big[\partial_\theta p_\theta(m)\Big]_{\theta=\theta_0} \hat\theta(m)=1.$$ The paper reports these in the multiparameter case, while I rewrote them for simplicity in the case of a single parameter. Here, $$p_\theta(m)$$ is the probability of the $$m$$-th outcome when the true parameter is $$\theta$$ (we're assuming discrete distributions), $$\hat\theta$$ is the estimator, and $$\theta_0$$ is the value of the true parameter wrt which the estimator is locally unbiased.
With these definitions in mind, the trivial constant estimator $$\hat\theta(m)=\theta_0$$ is not locally unbiased: while it obviously satisfies the first unbiasedness requirement, it doesn't satisfy the one with the derivative, as $$\sum_m \big[\partial_\theta p_\theta(m)\big]_{\theta=\theta_0} \hat\theta(m) = \theta_0\sum_m \big[\partial_\theta p_\theta(m)\big]_{\theta=\theta_0} = \theta_0 \bigg[\partial_\theta \sum_m p_\theta(m)\bigg]_{\theta=\theta_0}=0.$$