0
$\begingroup$

I'm not sure how to express the variance of this estimator. Here's the setup.

We have $X\sim N(0,\sigma^2)$ and want to estimate $\mathbb{E}[\phi(X)]$ where $\phi : \mathbb{R}\to\mathbb{R}$ is some function such that $\mathbb{E}[\phi(X)]$ has finite mean and variance. We have iid samples $Y_1,\dots, Y_n \sim N(0,1)$.

This is the estimator proposed:

$$\hat{\theta} = \frac{1}{n\sigma}\sum_{i=1}^{n} \exp \left[-Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i).$$

I have shown this estimator is unbiased. But I'm not sure how to express its variance. The most I can say is that since the $Y_i$ are iid, we have

$$var(\hat{\theta})=\frac{1}{n^2\sigma^2}\sum_{i=1}^{n} var\left(\exp \left[-Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i)\right).$$

How can I express this further?

$\endgroup$
9
  • 1
    $\begingroup$ Please start by checking your work, because for $\sigma \ne 1$ your $\hat\theta$ doesn't look in the least unbiased! Maybe there's a typographical error in your post? $\endgroup$
    – whuber
    Feb 12, 2023 at 15:21
  • $\begingroup$ @whuber Actually it is unbiased. I have done the calculation in a few lines. Why do you think it is not unbiased? $\endgroup$
    – jet
    Feb 12, 2023 at 15:24
  • 1
    $\begingroup$ Because (1) I cannot demonstrate its lack of bias and (2) with simulated data it is clearly biased. $\endgroup$
    – whuber
    Feb 12, 2023 at 16:37
  • $\begingroup$ 1. Your estimator is clearly biased. 2. If you want to estimate the expected value of a function of $X$, why not just calculate the sample mean of the function of $X$? How to get the standard error of this estimate should be obvious. $\endgroup$
    – jbowman
    Feb 12, 2023 at 16:57
  • $\begingroup$ @jbowman Pardon me, sirs, for persisting in my view of the unbiasedness of my estimator. I will present a few-lined proof in answer to this question so you can view it $\endgroup$
    – jet
    Feb 12, 2023 at 17:00

2 Answers 2

1
$\begingroup$

$\DeclareMathOperator{\Var}{Var}$

You can perform the similar calculation for variance by noting $\Var(Y) = E[Y^2] - (E[Y])^2$ for any random variable $Y$: \begin{align} & E\left[\exp \left[-2Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i)^2\right] \\ =& \int_{-\infty}^\infty \exp\left[ -y^2\left(\frac{1}{\sigma^2}-1\right)\right]\phi(y)^2 \cdot \frac{1}{\sqrt{2\pi}} \exp(-y^2/2)dy \\ =& \int_{-\infty}^\infty \phi(y)^2 \cdot \frac{1}{\sqrt{2\pi}}\exp\left(-y^2\left(\frac{1}{\sigma^2} - \frac{1}{2}\right)\right)dy \end{align}

Now here comes the problem, this integral diverges if $\sigma^2 > 2$. If $\sigma^2 \leq 2$, then you can further simplify by making a change of variable of $y$ to match the standard normal density $\varphi(x) = \frac{1}{\sqrt{2\pi}}e^{-x^2/2}$.

$\endgroup$
6
  • $\begingroup$ Thank you @Zhanxiong. Is there a weaker condition if we know that $\int_\infty^\infty \phi(x)^2 \, dx<\infty$? $\endgroup$
    – jet
    Feb 12, 2023 at 17:53
  • $\begingroup$ Won't work: for example, if $\sigma^2 = \frac{1}{4}$, then the dominant part of the integrand is $e^{y^2/2}$. $\phi(y) = 1/y^2$ satisfies $\int \phi(x)^2dx < \infty$ but the original integral is still divergent. $\endgroup$
    – Zhanxiong
    Feb 12, 2023 at 18:07
  • $\begingroup$ Do you mean if $\sigma^2=4$? $\endgroup$
    – jet
    Feb 12, 2023 at 18:16
  • $\begingroup$ Yes, it was a typo. $\endgroup$
    – Zhanxiong
    Feb 12, 2023 at 18:18
  • $\begingroup$ I see. Also, how does $\phi(y)=1/y^2$ satisfy $\int \phi(x)^2 \, dx <\infty$? It diverges, no? $\endgroup$
    – jet
    Feb 12, 2023 at 18:20
0
$\begingroup$

$$\mathbb{E}[\hat{\theta}]=\mathbb{E}\left[\frac{1}{n\sigma}\sum_{i=1}^{n} \exp \left[-Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i)\right]$$

$$ = \frac{1}{n}\sum_{i=1}^{n} \mathbb{E}\left[\frac{1}{\sigma}\exp \left[-Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i)\right]$$

Now, consider the expectation above. We have

$$\mathbb{E}\left[\frac{1}{\sigma}\exp \left[-Y_i^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(Y_i)\right] = \int_\infty^\infty \frac{1}{\sigma}\exp\left[ -y^2\left(\frac{1}{2\sigma^2}-\frac{1}{2}\right)\right]\phi(y) \cdot\frac{1}{\sqrt{2\pi}} \exp(-y^2/2) \, dy$$

$$= \int_\infty^\infty \phi(y) \cdot \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{1}{2}\left(\frac{y}{\sigma}\right)^2\right) \, dy$$

$$ = \mathbb{E}[\phi(X)]$$ QED

$\endgroup$
8
  • 1
    $\begingroup$ How do you get from the next-to-last line to the last line, given that $Y$ is not in fact distributed $N(0, \sigma^2)$? $\endgroup$
    – jbowman
    Feb 12, 2023 at 17:19
  • $\begingroup$ @jbowman $y$ in the integration is a dummy variable! $\endgroup$
    – jet
    Feb 12, 2023 at 17:26
  • 1
    $\begingroup$ ... which fails to address the point. You are integrating $\phi(y)$ with respect to a density function which is not the actual distribution of $y$, so you won't get the expected value of $\phi(y)$ out (unless, as @whuber pointed out, $\sigma = 1$.) You will get the expected value when $Y \sim N(0, \sigma^2)$, but you have already specified that $Y \sim N(0,1)$. $\endgroup$
    – jbowman
    Feb 12, 2023 at 18:13
  • $\begingroup$ @jbowman $\phi$ is just a function on $\mathbb{R}$. We don't integrate "wrt a density function", we integrate wrt a real variable traversing $\mathbb{R}$. $\endgroup$
    – jet
    Feb 12, 2023 at 18:28
  • 1
    $\begingroup$ 1. Yes we do, actually: math.stackexchange.com/questions/380785/… may get you started on this more advanced approach! 2. $\mathbb{E}\phi(y) = \int_{-\infty}^{\infty}\phi(y)f(y)dy$, where $f(y)$ is the probability density function of $y$. Where is that probability density, the $N(0,1)$ specified above, in your next-to-last equation? $\endgroup$
    – jbowman
    Feb 12, 2023 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.