Where does the efficient locally unbiased estimator $\tilde\theta(m)=\theta_0+\frac{F^{-1}\nabla p_\theta(m)}{p_\theta(m)}$ come from?

Question

In Equation (9), page 9 of (Demkowicz-Dobrzanski et al. 2020), the authors mention that, given a probability distribution $p_{\boldsymbol\theta}(m)$ with hidden parameter $\boldsymbol\theta$ and $m$ labelling the possible outcomes, if the true parameter $\boldsymbol\theta$ is close to some known $\boldsymbol\theta_0$, then we have a locally unbiased estimator $\tilde{\boldsymbol\theta}$ that saturates the CR bound and is thus optimal at $\boldsymbol\theta_0$, written as $$\tilde{\boldsymbol\theta}(m) = \boldsymbol\theta_0 + \frac{1}{p_{\boldsymbol\theta}(m)} F^{-1} \nabla p_{\boldsymbol\theta}(m)\big|_{\boldsymbol\theta=\boldsymbol\theta_0},\tag1$$ where $F$ is the Fisher information matrix: $$F = \sum_m \frac{\nabla p_{\boldsymbol\theta}(m)[\nabla p_{\boldsymbol\theta}(m)]^T}{p_{\boldsymbol\theta}(m)}.$$

That this estimator is efficient around the true value I can see because $$\mathbb{E}[(\tilde{\boldsymbol\theta}-\boldsymbol\theta_0)^2] = \sum_m \frac{1}{p(m)}\sum_{ijk} (F^{-1})_{ij}(F^{-1})_{ik} \partial_j p(m)\partial_k p(m) \\ = \sum_i (F^{-1}FF^{-1})_{ii} = \operatorname{tr}(F^{-1}).$$ What I'm not clear about is how (1) is derived in the first place. Sure, once I have it I can verify that it works, but what's a way to get to it from scratch? I thought of trying to find the locally unbiased estimator that minimises the variance at the true value, but that would just give me back the trivial estimator $m\mapsto \boldsymbol\theta_0$.

Is this particular structure characterised by its being locally unbiased and efficient? It would seem like that's what is being stated when it's discussed in the paper. But isn't the trivial estimator $\tilde{\boldsymbol\theta}(m)=\boldsymbol\theta_0$ already enough for that? it's locally unbiased and has zero variance around the true parameter. So how is this one better exactly?

Answer 1

This is not a full answer, but it addresses one issue in the original question: I misunderstood the definition of "locally unbiased", and using the correct definition the trivial constant estimator doesn't qualify. The question of how to derive the locally unbiased estimator built via the Fisher still stands, though.

---

As pointed out in the comments, I seem to have misunderstood what the authors mean with "locally unbiased". I thought it just meant to have an estimator that is unbiased when the true parameter has a specific value. However, the paper also requires a stronger condition, which I understand as asking the estimator to also be "well-behaved" around the parameter value where it's unbiased (also, I couldn't find this definition of "locally unbiased" in other recent sources, so it might not be that common in the statistical literature).

More precisely, we're asking the two conditions, spelled out in Eqs. (4) and (5) in the above linked paper: $$\sum_m p_{\theta_0}(m)\hat\theta(m)=\theta_0, \qquad \sum_m \Big[\partial_\theta p_\theta(m)\Big]_{\theta=\theta_0} \hat\theta(m)=1.$$ The paper reports these in the multiparameter case, while I rewrote them for simplicity in the case of a single parameter. Here, $p_\theta(m)$ is the probability of the $m$-th outcome when the true parameter is $\theta$ (we're assuming discrete distributions), $\hat\theta$ is the estimator, and $\theta_0$ is the value of the true parameter wrt which the estimator is locally unbiased.

With these definitions in mind, the trivial constant estimator $\hat\theta(m)=\theta_0$ is not locally unbiased: while it obviously satisfies the first unbiasedness requirement, it doesn't satisfy the one with the derivative, as $$\sum_m \big[\partial_\theta p_\theta(m)\big]_{\theta=\theta_0} \hat\theta(m) = \theta_0\sum_m \big[\partial_\theta p_\theta(m)\big]_{\theta=\theta_0} = \theta_0 \bigg[\partial_\theta \sum_m p_\theta(m)\bigg]_{\theta=\theta_0}=0.$$