Implicit hypothesis testing: mean greater than variance and Delta Method

Question

I am struggling with a hypothesis test between the mean and variance of a sample of i.i.d Gaussian random variables. This (self-study) question arises in the context of the Delta Method (first or second order) and constructing asymptotic test statistics:

Based on a sample of i.i.d. Gaussian random variables $X_1, . . . , X_n$ with mean $\mu$ and variance $\sigma^2$, propose a test with asymptotic level 5% for the hypotheses

$H_0 : \mu >\sigma \ vs \ H_1 : \mu \leq \sigma$

Potential hints from the lecture notes:

Given a hypothesis of the form: $H_0: g(\theta) = 0$ where $g: \mathbb{R^d} \to \mathbb{R^k}$

We are given:

• If the MLE estimator is asymptotically normal: $\sqrt{n} (\hat \theta - \theta) \xrightarrow{d} N_d(0,\Sigma(\theta))$
• The first order delta method gives: $\sqrt{n} (g(\hat \theta) - g(\theta)) \xrightarrow{d} N_k(0,\Gamma(\theta))$ which can be rewritten as $\sqrt{n} \ \ \Gamma(\theta)^{-1/2} (g(\hat \theta) - g(\theta)) \xrightarrow{d} N_k(0,I_k)$
  • $\Gamma (\theta) = \nabla g(\theta)^T \Sigma(\theta)\nabla g(\theta) \in \mathbb{R}^{k \times k}$
  • Assume $\Sigma$ is invertible, $\nabla g$ has rank $k$, $\Gamma$ is invertible and continuous in $\theta$
• By Slutsky's theorem we can replace $\theta$ with the MLE $\hat \theta$ such that $\sqrt{n} \ \Gamma (\hat \theta)^{-1/2} \ (g(\hat \theta) - g(\theta)) \xrightarrow{d} N_k(0,I_k)$
• Second order delta method: $n \ g(\hat \theta)^T \ \Gamma \ (\hat \theta)^{-1} g(\theta) \ \xrightarrow{d} \chi^2_k$

First attempt

We can reformulate the null hypothesis as: $H_0 : g(\mu, \sigma): \mu - \sqrt{\sigma^2} < 0$ which maps from $\mathbb{R^2} \to \mathbb{R}$ so $k = 1$

\begin{aligned} & \frac{\partial g}{\partial \mu} = 1 \\ & \frac{\partial g}{\partial \sigma^2} = - \frac{1}{2\sigma} \\ & \nabla g(\mu, \sigma) = [1,- \frac{1}{2\sigma}]^T \end{aligned}

The MLE estimators for the normal distribution are:

• $\hat \mu = \bar{X}_n$ with asymptotic distribution $\xrightarrow{d} N(\mu, \frac{\sigma^2}{n})$
• $\hat \sigma^2 = \frac{1}{n} \sum X_i - \bar{X}_n^2$ with asymptotic distribution $\xrightarrow{d} N(\sigma^2, \frac{2\sigma^4}{n})$

The covariance matrix $\Sigma$ for the sample of i.i.d. normal r.v. can be found from the Fisher Matrix or otherwise:

$\Sigma(\mu, \sigma^2) = \begin{bmatrix}\frac{\sigma^2}{n} & 0 \\0 & \frac{2 \sigma^4}{n} \end{bmatrix}$

Computing $\Gamma$ which should be a $1 \times 1$ matrix as $k =1$ \begin{aligned} \Gamma &= \nabla g^T \Sigma \nabla g \\ & = [1,- \frac{1}{2\sigma}] \ \begin{bmatrix}\frac{\sigma^2}{n} & 0 \\0 & \frac{2 \sigma^4}{n} \end{bmatrix} \ [1,- \frac{1}{2\sigma}]^T \\ & = \frac{3 \sigma^2}{n} \\ \Gamma^{-1/2} & = \frac{\sqrt{n}}{\sigma \sqrt{3}} \end{aligned}

Computing the test statistic based on the first order Delta Method (i.e. normal distribution) assuming that under $H_0 : g(\mu, \sigma) = 0$ (How can we deal with the inequality ?)

\begin{aligned} T_n & = \sqrt{n} \ \ \Gamma(\theta)^{-1/2} (g(\hat \theta) - g(\theta)) \\ & = \sqrt{n} \ \frac{\sqrt{n}}{\hat \sigma \sqrt{3}} [ (\hat \mu - \hat \sigma) - 0 ] \\ & = \ \frac{n (\hat \mu - \hat \sigma)}{\hat \sigma \sqrt{3}} \end{aligned}

Computing the second order delta method test statistic (i.e. chi-squared distribution)

\begin{aligned} T_n & = n \ \ g(\hat \theta)\ \ \Gamma(\theta)^{-1}\ g(\hat \theta) \\ & = n (\hat \mu - \hat \sigma) \frac{n}{3 \sigma^2} (\hat \mu - \hat \sigma) \\ & = \frac{n^2 (\hat \mu - \hat \sigma)}{3 \hat \sigma^2} \end{aligned}

Questions

• Is this the right way to deal with such a hypothesis test ?
• What distinguishes the first order test statistic (normal distribution) from the second order test statistic (Chi-squared distribution)
• What happens to the inequality sign ?

Answer 1

What distinguishes the first order test statistic (normal distribution) from the second order test statistic (Chi-squared distribution)

With the first order approximation you approximate the function $g(\theta)$ as a linear function of $\theta$. But this works only when there is actually a slope, that is when $g(\theta)' \neq 0$.

With the second order approximation you approximate the function $g(\theta)$ as a polynomial function (the square) of $\theta$, but this works only in a peak of the function $g(\theta)$.

I do not believe that this is applicable to your case and that you applied it correctly (It seems like you just took the square of the first order).

The image below might illustrate this intuitively:

In this example $Y=0.03 X^2$. And $X \sim N(20, \sigma)$ is normal distributed with $\sigma$ changing from $36$ to $4$ and $1$. Simulations are made for 600 data points to create the histograms (60 points are used to plot on top of the curve in the graph). In the image on the left, when $X$ has a wide distribution, you see that the distribution of $Y$ is not well approximated with a linear transformation (it is a bit skewed), but as the variance of $X$ decreases (the images on the right) then the distribution starts to resemble more and more a normal distribution.

So that is what the linear transformation does in the Delta method. But when the slope is zero then this linearization doesn't work and you need to use a second order approximation of the curve. This is illustrated below

What happens to the inequality sign ?

With $H_0 : \mu >\sigma$ you have a composed hypothesis instead of a simple hypothesis $H_0 : \mu = \sigma$. This is not easy to deal with and you will typically not be able to find a hypothesis test where the probability for type I error is equal for every value of parameters that are possible in the null hypothesis.

In this case when you use the boundary for $H_0 : \mu = \sigma$ then you will have a rejection (type I error) rate $\alpha$ when the hypothesis is true $\mu = \sigma$ but you get smaller rejection rates when $\mu>\sigma$.

---

Is this the right way to deal with such a hypothesis test ?

The Delta method is very easy to apply. But in this case you could also consider the statistic $T = \sqrt{n}\frac{\bar{x}}{s}$ which follows a non-central t-distribution with non centrality parameter $\sqrt{n}\frac{\mu}{\sigma}$.

Then you can use code for computing the non central t-distribution to compute boundaries more precisely than the delta method approximation.

# testing performance of statistic mean(y)/sd(y)
# in comparison to non-central t-distribution

set.seed(1)

n = 5
mu = 3
sigma = 3

dt <- 0.2 # historgram binsize

# doing simulations
mc_test <- sapply(1:10^6, FUN = function(x) {y <- rnorm(n,mean=mu,sd=sigma); sqrt(n)*mean(y)/sd(y)})

# computing and plotting histogram
h <- hist(mc_test,
          breaks=seq(min(mc_test)-dt,max(mc_test)+dt,dt),
          xlim=c(-3,10),
          freq = FALSE,
          ylab = bquote(t-dist(T, nu == .(n), ncp==1)),
          xlab = bquote(T == bar(x)/s),
          main = "histogram of simulations compared with non-central t-distribution", cex.main=1
          )

# adding non central t-distribution to the plot
t <- seq(-3,10,0.01)
lines(t,dt(t,n-1,sqrt(n)),col=2)

ts <- seq(qt(0.95,n,sqrt(n)),10,0.01)
polygon(c(rev(ts),ts),c(0*dt(ts,n-1,sqrt(n)),dt(ts,n-1,sqrt(n))),  
        col = rgb(0,0,0,0.3), border = NA)

# verify/compute how often boundary is exceeded
sum(mc_test>qt(0.95,n-1,sqrt(n)))/10^6

Comparison of boundaries as function of the sample size $n$

Your statistic* $\frac{\sqrt{n} (\hat \mu - \hat \sigma)}{\hat \sigma \sqrt{1.5}} \sim N(0,1)$ leads to $\sqrt{n}\frac{\bar{x}}{s} \sim N(\sqrt{n},1.5)$ which is the asymptotic behaviour of the non central distribution that we derived:

#different values of n
n <- 3:200
# boundary based on t-distribution
bt <- qt(0.95,n-1,sqrt(n))
#boundary based on delta method
dt <- qnorm(0.95,sqrt(n),sqrt(1.5))

# plotting
plot(n,dt,type='l',
     xlab = "n",ylab = "95% criterium" )
lines(n,bt,pch=21,col=2)

legend(0,16,c("t-distribution", "Delta-method"),box.col=0,col=c(1,2),lty=1,cex=0.7)

---

*In your computations the factor $3$ should be a factor $1.5$

$$[1,- \frac{1}{2\sigma}] \ \begin{bmatrix}\frac{\sigma^2}{n} & 0 \\0 & \frac{2 \sigma^4}{n} \end{bmatrix} \ [1,- \frac{1}{2\sigma}]^T = \frac{1.5 \sigma^2}{n}$$

and also the square root term $\sqrt{n}$ should not be added (because you only have one measurement of $\hat\mu - \sqrt{\hat\sigma^2}$ ). You used a formula for the Delta method that incorporates a term $\sqrt{n}$ for multiple measurements, but you already accounted for multiple measurements when you expressed the variance of $\hat\mu$ and $\hat\sigma^2$.

Answer 2

Few notes:

1)In last eqaution $(\hat \mu - \hat \sigma)$ should be squared.

2)Cov. matrix $\Sigma(\mu, \sigma^2) $ shouldn't depend on $n$, it is asymptotic entity. So as $\Gamma$ which in this case is equal $\Gamma = \frac{3 \sigma^2}{2}$.

So the final answer is $T_n = \frac{2n^2 (\hat \mu - \hat \sigma)^2}{3 \hat \sigma^2}$