Importance sampling estimation of power function

Question

Problem

Suppose we are given $\text{Poisson}(\theta)$ model, and the null hypothesis is as follows:

$$ H_0 : \theta = 0.1 \ \ \text{vs}. \ \ H_1 : \theta < 0.1 $$

Suppose we take sample of $n=100$ from the model, $X_1, \ldots, X_{100}$.

Plot the power function $\gamma(\theta)$, with standard error, of the following test, using importance sampling.

Test : reject $H_0$ at $\alpha=0.05$, when

$$ \frac{\bar{X} - 0.1}{\sqrt{0.1/100}} < -1.645 $$

In the grammar of test function,

$$ \phi(X_1, \ldots, X_{100}) = \begin{cases} 1 & \mathrm{if} \frac{\bar{X} - 0.1}{\sqrt{0.1/100}} < -1.645 \\[7pt] 0 & \mathrm{if} \frac{\bar{X} - 0.1}{\sqrt{0.1/100}} \ge -1.645 \\[7pt] \end{cases} $$

---

Try

Power function is defined as

$$ \gamma(\theta) := \mathrm{E} \left[ \phi(X_1, \ldots, X_{100}) \right] $$

for a fixed $\theta$.

So my strategy is to first fix $\theta$, and evaluate the quantity

$$ \mathrm{E} \left[ \phi(X_1, \ldots, X_{100}) \right] = \Pr\left(\frac{\bar{X} - 0.1}{\sqrt{0.1/100}} < -1.645 \right) $$

where $100\bar{X} \sim \text{Poisson}(100\theta)$.

---

Question

I do not see any place to apply the importance sampling technique. How the importance sampling (which can give us standard error) can contribute my question?

Any help will be appreciated.

Answer 1

I will mention three ways to approximate the following $$ \mathrm{E} \left[ \phi(X_1, \cdots, X_{100}) \right] = Pr\left(\frac{\bar{X} - 0.1}{\sqrt{0.1/100}} < -1.645 \right). $$

1. Use the CLT to justify $$ \Phi(-1.645). $$

2.

Second, simulate $N$ length $100$ data sets from your joint mass function to calculate $$ \frac{1}{N}\sum_{i=1}^N \phi(X^i_1, \cdots, X^i_{100}) $$

3. Simulate $N$ length $100$ data sets from some proposal pmf $q(x_1, \ldots, x_{100})$ and calculate $$ \frac{1}{N}\sum_{i=1}^N\phi(X^i_1, \cdots, X^i_{100}) \frac{p(x^i_1, \ldots, x^i_{100})}{q(X^i_1, \cdots, X^i_{100})} $$ where $p$ is your true product-Poisson pmf.

Calculating approximate standard errors is probably easier for the last two, but you may also get Berry-Esseen bounds for the first one.

To check your answers, it might also be worth evaluating the true probability. This is possible using what you mentioned: that the sum of iid Poissons is also Poisson-distributed. The cdf is available in most statistical software packages: $$ Pr\left(\sum_j X_j < 10 -1.645\sqrt{10} \right). $$

Answer 2

Here is a power curve for an exact test of $H_0: \lambda = 0.1$ vs. $H_a: \lambda \le 0.1,$ based on $n = 100$ observations from $\mathsf{Pois}(\lambda).$

Under $H_0,$ the total $T$ of the $n = 100$ observations $X_i \sim \mathsf{Pois}(0.1)$ has $T \sim \mathsf{Pois}(10).$ Because the Poisson distribution is discrete, a (nonrandomized) test at exactly level $\alpha = 0.05$ is not available.

The largest available level below 5% is $\alpha = 0.0293.$ So we will find the power of a test that rejects $H_0$ when $T \le 4$ or $\bar X = T/n \le 0.04.$ (See computations in R below.)

qpois(.05, 10)
[1] 5
ppois(5,10)
[1] 0.06708596
ppois(4,10)
[1] 0.02925269

The power of the test against alternative $\lambda_0 < 0.1$ is $P(T \le 4\,|\,\lambda_0).$

Here is a graph of the power function:

lam = seq(10, .01, by=-.01)/100
p.rej = ppois(4, 100*lam)
plot(lam, p.rej, type="l", ylim=0:1, main="Power Curve")
 abline(h=0:1, col="green2");  abline(v=c(0,.1), col="green2")