1
$\begingroup$

A standard problem in classical statistics is to find a good estimator that minimizes a given cost function under certain conditions. Normally we want to require the estimator $\hat\theta$ to be unbiased in every possible value of unknown parameters, that is $$ \mathrm{E}_{\boldsymbol{\theta }}^{(n)}\left[ \hat{\theta}\left( X^n \right) \right] :=\sum_{x^n\in \mathcal{X} ^n}{p_{\boldsymbol{\theta }}^{(n)}}\left( x^n \right) \hat{\theta}\left( x^n \right) =\theta $$ where $x^n$ is the experiment result for $n$ times. In eq(5) of this paper, the author states that we can get the following local unbiased condition by Taylor expanding the above conditions to be $$ \begin{array}{c} \mathrm{E}_{\boldsymbol{\theta }}^{(n)}\left[ \hat{\theta}\left( X^n \right) \right] =\sum_{x^n\in \mathcal{X} ^n}{p_{\boldsymbol{\theta }}^{(n)}}\left( x^n \right) \hat{\theta}\left( x^n \right) =\theta ,\\ \partial _{\theta}\mathrm{E}_{\boldsymbol{\theta }}^{(n)}\left[ \hat{\theta}\left( X^n \right) \right] =\sum_{x^n\in \mathcal{X} ^n}{\partial _{\theta}}p_{\boldsymbol{\theta }}^{(n)}\left( x^n \right) \hat{\theta}\left( x^n \right) =1,\\ \end{array} $$ But I'm not sure how can we do the Taylor expansion to get the second set of restrictions.

$\endgroup$

2 Answers 2

1
$\begingroup$

Work on local unbiasedness was done back in the 50s. The most prominent and accessible work $[\rm I]$ is due to Fraser.

First, understand the definition: $h(\mathbf x) $ is locally unbiased at $\boldsymbol\theta_0$ if

\begin{align}\mathbb E[h(\mathbf x) \mid \boldsymbol\theta_0]&=\boldsymbol\theta_0,\tag{LU-1}\label 2\\ \frac{\partial}{\partial\boldsymbol\theta_0}\mathbb E[h(\mathbf x) \mid \boldsymbol\theta_0]&=\mathbf I. \tag{LU-2}\label 1\end{align}

A globally-unbiased estimator satisfies the conditions for every $\boldsymbol\theta_0.$ Implicit assumption is the basic regularity condition: the support should not depend on $\boldsymbol\theta$ at $\boldsymbol\theta_0$ and interchange of derivative and integral is valid.

For simplicity consider one-dimensional parameter $\theta.$ Then $h(\mathbf x) $ is unbiased locally at $\theta_0$ if $\eqref{2}, \eqref{1}$ hold. But this just implies $$\mathbb E[h(\mathbf x) \mid \theta]=\theta_0+(\theta-\theta_0)+o(\theta-\theta_0).$$

Can you now be able to connect to and show your result?

By expanding the log-likelihood at nhood of $\theta_0, $ and using a minimal-sufficient statistic based on it, one can construct using the CR-bound a local-unbiased estimator.


Reference:

$\rm [I]$ On Local Unbiased Estimation, D. A. S. Fraser, Journal of the Royal Statistical Society. Series B (Methodological), Vol. $26,$ No. $1,~ (1964),$ pp.$ 46-51.$

$\endgroup$
0
$\begingroup$

The way that I understand how the local unbiased condition is obtained from the global unbiased condition can well be shown in the following picture enter image description here

where the horizontal axis means the true value $\theta$ and the vertical axis means the expectation value of the estimator $\hat \theta$. The mean value of our estimator should be exactly the true value at the true value point($1/2$ in the picture) and the first derivative of the red curve should have the same slope as the blue one in the true value point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.