8
$\begingroup$

I have a parameter $\theta$ which lies between $[0,1]$. Let us say that I can run an experiment and obtain $\hat{\theta} = \theta + w$, where $w$ is a standard Gaussian. What I need is an estimate of $\theta$ which is 1) unbiased 2) almost surely bounded. Requirement (2) is crucial for me.

The natural think to do is to construct a new estimator setting $\hat{\theta}$ to $1$ if it is above $1$ and to $0$ if it is below $0$. But then the estimator will not be unbiased. So what should I do?

Formally, the question is whether there exists a function $f: \mathbb{R} \rightarrow \mathbb{R}$ such that $f(\hat \theta)$ satisfies (1) and (2) above. Moreover, would the situation be different if I drew more than a single sample?

$\endgroup$
4
  • $\begingroup$ Can you say more about your situation? I'm not a mathematical statistician, but this seems very abstract to me. It does remind me of logistic regression, where the parameter $\pi$ must lie in $(0,1)$, and $E[\hat\pi]=\pi$, but the sampling distribution of $\hat\pi$ is not Gaussian. (Of course, $\text{logit}(\hat\pi)$ is, but then that's not bounded by $(0,1)$.) Is any of that related to your situation? FWIW, I suspect you won't be able to find a function like you want (ie, that's bounded), b/c $\mathbb R$ is not bounded. (W/ apologies, I can delete this comment if needed.) $\endgroup$ Jun 10, 2013 at 4:55
  • $\begingroup$ I agree that it most likely that no such function $f$ exists, even if we expand the condition to collecting multiple samples. If that is the case, though, I'd still be interested in seeing a proof that no such function exists. $\endgroup$
    – yves
    Jun 10, 2013 at 5:22
  • 2
    $\begingroup$ The expression $\hat{\theta} = \theta + w$ is a theoretical expression to which one usually arrives while trying to determine the properties of the estimator, unbiasedness in this case. But this is not the actual functional form of the estimator, because it contains the unknown parameter $\theta$. In order to meaningfully explore your question, we need the expression of $\hat \theta$ as a function of the data. This cannot be answered in general. $\endgroup$ Jan 26, 2014 at 1:55
  • $\begingroup$ I have the same question! More precisely, the question is whether there exist $-\infty< a < b < \infty$ and a measurable function $f : \mathbb{R} \to [a,b]$ such that $$\forall \mu \in [0,1] ~~~~~ \underset{X \leftarrow \mathcal{N}(\mu,1)}{\mathbb{E}}[ f(X) ] = \mu.$$ I believe the answer is no, but I'm looking for a proof that no such $f$ exists. $\endgroup$
    – Thomas
    Nov 2, 2018 at 19:24

1 Answer 1

3
$\begingroup$

I will present conditions under which an unbiased estimator remains unbiased, even after it is bounded. But I am not sure that they amount to something interesting or useful.

Let an estimator $\hat \theta$ of the unknown parameter $\theta$ of a continuous distribution, and $E(\hat \theta) =\theta$.

Assume that for some reasons, under repeated sampling we want the estimator to produce estimates that range in $[\delta_l,\delta_u]$. We assume that $\theta \in [\delta_l,\delta_u]$ and so we can write when convenient the interval as $[\theta-a,\theta+b]$ with $\{a,b\}$ positive numbers but of course unknown.

Then the constrained estimator is

$$\hat \theta_c = \left\{\begin{matrix} \delta_l & \hat \theta <\delta_l\\\hat \theta &\delta_l \leq \hat \theta \leq \delta_u \\\delta_u & \delta_u < \hat \theta \end{matrix} \right\}$$

and its expected value is

$$\begin{align} E(\hat \theta_c) &= \delta_l\cdot P[\hat \theta \leq \delta_l] \\&+ E(\hat \theta \mid \delta_l \leq\hat \theta \leq\delta_u )\cdot P[\delta_l \leq\hat \theta \leq \delta_u] \\ &+\delta_u\cdot P[\hat \theta > \delta_u]\end{align}$$

Define now the indicator functions

$$I_l = I(\hat \theta \leq \delta_l),\;\; I_m = I(\delta_l\leq \hat \theta \leq \delta_l),\;\; I_u = I(\hat \theta > \delta_u)$$

and note that

$$I_l + I_u = 1- I_m \tag{1}$$

using these indicator functions, and integrals, we can write the expected value of the constrained estimator as ($f(\hat \theta)$ is the density function of $\hat \theta$),

$$E(\hat \theta_c) = \int_{-\infty}^{\infty}\delta_lf(\hat \theta)I_ld\hat \theta + \int_{-\infty}^{\infty}\hat \theta f(\hat \theta)I_md\hat \theta + \int_{-\infty}^{\infty} \delta_uf(\hat \theta)I_ud\hat \theta$$

$$=\int_{-\infty}^{\infty}f(\hat \theta)\Big[\delta_lI_l + \hat \theta I_m + \delta_uI_u\Big]d\hat \theta$$

$$=E\Big[\delta_lI_l + \hat \theta I_m + \delta_uI_u\Big] \tag{2}$$

Decomposing the upper and lower bound, we have

$$E(\hat \theta_c) = E\Big[(\theta-a)I_l + \hat \theta I_m + (\theta+b)I_u\Big]$$

$$=E\Big[\theta\cdot(I_l+I_u) + \hat \theta I_m\Big] -aE(I_l)+bE(I_u) $$

and using $(1)$,

$$ = E\Big[\theta\cdot(1-I_m) + \hat \theta I_m\Big] -aE(I_l)+bE(I_u) $$

$$\Rightarrow E(\hat \theta_c) = \theta +E\big[(\hat \theta -\theta)I_m\big]-aE(I_l)+bE(I_u) \tag {3}$$

Now, since $E(\hat \theta) = \theta$ we have

$$E\big[(\hat \theta -\theta)I_m\big] = E\big(\hat \theta I_m\big) - E(\hat \theta)E(I_m)$$

But

$$E\big(\hat \theta I_m\big) = E\big(\hat \theta I_m\mid I_m=1\big)E(I_m) = E\big(\hat \theta \big)E(I_m)$$

Hence, $E\big[(\hat \theta -\theta)I_m\big] =0$ and so

$$\begin{align} E(\hat \theta_c) &= \theta -aE(I_l)+bE(I_u) \\ &= \theta -aP(\hat \theta \leq \delta_l)+bP(\hat \theta > \delta_u)\end{align}\tag {4}$$

or alternatively

$$ E(\hat \theta_c) = \theta -(\theta-\delta_l)P(\hat \theta \leq \delta_l)+(\delta_u-\theta)P(\hat \theta > \delta_u)\tag {4a}$$

Therefore from $(4)$, we see that for the constrained estimator to also be unbiased, we must have

$$aP(\hat \theta \leq \delta_l) = bP(\hat \theta > \delta_u) \tag {5}$$

What is the problem with condition $(5)$? It involves the unknown numbers $\{a,b\}$, so in practice we will not be able to actually determine an interval to bound the estimator and keep it unbiased.

But let's say this is some controlled simulation experiment, where we want to investigate other properties of estimators, given unbiasedness. Then we can "neutralize" $a$ and $b$ by setting $a=b$, which essentially creates a symmetric interval around the value of $\theta$... In this case, to achieve unbiasedness, we must more over have $P(\hat \theta \leq \delta_l) = P(\hat \theta > \delta_u)$, i.e. we must have that the probability mass of the unconstrained estimator is equal to the left and to the right of the (symmetric around $\theta$) interval...

...and so we learn that (as sufficient conditions), if the distribution of the unconstrained estimator is symmetric around the true value, then the estimator constrained in an interval symmetric around the true value will also be unbiased... but this is almost trivially evident or intuitive, isn't it?

It becomes a little more interesting, if we realize that the necessary and sufficient condition (given a symmetric interval) a) does not require a symmetric distribution, only equal probability mass "in the tails" (and this in turn does not imply that the distribution of the mass in each tail has to be identical) and b) permits that inside the interval, the estimator's density can have any non-symmetric shape that is consistent with maintaining unbiasedness -it will still make the constrained estimator unbiased.

APPLICATION: The OP's case
Our estimator is $\hat \theta = \theta + w,\;\; w \sim N(0,1)$ and so $\hat \theta \sim N(\theta,1)$. Then, using $(4)$ while writing $a,b$ in terms of $\theta, \delta$, we have, for bounding interval $[0,1]$,

$$E[\hat \theta_c] = \theta -\theta P(\hat \theta \leq 0) +(1-\theta)P(\hat \theta > 1)$$

The distribution is symmetric around $\theta$. Transforming ($\Phi()$ is the standard normal CDF)

$$E[\hat \theta_c] = \theta -\theta P(\hat \theta-\theta \leq -\theta) +(1-\theta)P(\hat \theta -\theta > 1-\theta)$$

$$=\theta -\theta \Phi(-\theta) +(1-\theta)[1-\Phi(1-\theta)]$$

One can verify that the additional terms cancel off only if $\theta =1/2$, namely, only if the bounding interval is also symmetric around $\theta$.

$\endgroup$
5
  • $\begingroup$ I don't think this answers the question. You're analyzing truncation. The question is not "Does truncation work?", but rather "Is there an alternative to truncation that does work?". OP seems to be aware that truncation does not work. $\endgroup$
    – Thomas
    Nov 2, 2018 at 19:17
  • $\begingroup$ @Thomas The OP asks (last sentence of the OP's post) whether we can have a bounded estimator that it is also unbiased. I present first a general treatment of the matter and then an application directly on the OP's premises. I don't understand why this "does not answer the question". $\endgroup$ Nov 2, 2018 at 20:43
  • $\begingroup$ You are assuming a specific functional form for the estimator, namely $$f(\hat \theta) = \left\{\begin{matrix} \delta_l & \hat \theta <\delta_l\\\hat \theta &\delta_l \leq \hat \theta \leq \delta_u \\\delta_u & \delta_u < \hat \theta \end{matrix} \right\}$$ for some $\delta_l,\delta_u \in \mathbb{R}$. My interpretation is that the question asks about any bounded estimator $f$, not just estimators with this functional form. For example, $f(\hat \theta) = \sin(\hat \theta)$ would be a bounded estimator (not a useful one though). $\endgroup$
    – Thomas
    Nov 2, 2018 at 22:32
  • $\begingroup$ (I'm commenting on this years-old question because I have the same question. In particular, the question I'm interested in is for arbitrary bounded estimators.) $\endgroup$
    – Thomas
    Nov 2, 2018 at 22:36
  • $\begingroup$ @Thomas True that my explorations does not treat boundedness in its outmost generality. True also that once you compose the estimator with a non-linear function, in general it must be on its own biased, as a necessary condition for the transformation to be unbiased. $\endgroup$ Nov 3, 2018 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.