1
$\begingroup$

enter image description here

Here under $H_0:P_1=P_2$ the standard error gets some common $P$ value which is computed from a pooled estimate. Why is the $\sigma$ value not calculated in a similar way in the next one? There under null $H_0:\sigma_1=\sigma_2$ why that is not substituted in the standard error formula?

enter image description here

$\endgroup$
  • $\begingroup$ Hello. Can you please specify what $p_1, p_2, q_1, q_2$ are in the 1st image ? Also, what was being tested there ? (By the way, which book is this ?) $\endgroup$ – Kolmogorov Nov 25 '20 at 4:55
  • $\begingroup$ Here $p_1,p_2,q_1,q_2$ are sample proportions, where $q_1=1-p_1$. In the first image it’s the difference of two sample proportions and the second image difference of two sample standard deviations. What I’m not getting is that why $\sigma_1=\sigma_2$ is not made use of like $P$ in the first one. $\endgroup$ – danny Nov 25 '20 at 5:35
  • 1
    $\begingroup$ Which book is this ? $\endgroup$ – Kolmogorov Nov 25 '20 at 7:44
  • 1
    $\begingroup$ Is this a question from a course or textbook? If so, please add the self-study tag & read its wiki. $\endgroup$ – kjetil b halvorsen Nov 26 '20 at 11:34
1
$\begingroup$

In a nutshell: I believe the title of your question might sound confusing to some readers, but the answer nevertheless can be "yes", to a possibly slightly modified question:

"Is it possible to use the parameter values specified in the null hypothesis in order to provide a valid estimate of a standard error?".

The second screenshot (if possible, try to avoid these and typeset the text in TeX to make the site more searchable) still is "incomplete" in that the last S.E. still depends on unknown quantities, viz. $\sigma_1$ and $\sigma_2$.

Hence, $S.E.(s_1-s_2)$ will have to be replaced with some estimator thereof, call it $\widehat{S.E.}(s_1-s_2)$, in order to get a test statistic, call it $\hat{Z}$ (recall that statisticians call a statistic something that we can actually compute, that does not depend on unknowns).

If that estimator is consistent for $S.E.(s_1-s_2)$ we obtain, by Slutzky's Lemma, $$ \hat{Z}=\frac{s_1-s_2}{\widehat{S.E.}(s_1-s_2)}=\underbrace{\underbrace{\frac{s_1-s_2}{S.E.(s_1-s_2)}}_{=Z\to_dN(0,1)}\cdot\underbrace{\frac{S.E.(s_1-s_2)}{\widehat{S.E.}(s_1-s_2)}}_{\to_p1}}_{=Z\to_dN(0,1)} $$

Now, you could do two things:

  1. Replace $\sigma_j^2$, $j=1,2$, in $S.E.(s_1-s_2)$ with $s_j^2$. That will result in a consistent estimator of $S.E.(s_1-s_2)$ both when the null is true or not.
  2. Estimate a common value $s^2$ from the full sample and plug that in for both $\sigma_j^2$, $j=1,2$, in $S.E.(s_1-s_2)$ (a "pooled" estimate).

For 2., if the null is true, the above result in the second term of the above display still holds and nothing changes, as estimating a common value was correct as both populations share the same standard deviation. Hence, the test statistic will still behave like a normal random variable in large samples if the null is true, such that we may employ the normal critical values to arrive at a test decision.

Now, if the null is false, what do we "want" from our test statistic $\hat{Z}$? We want it to be larger than the critical values of the test as often as possible ("high power").

Now, the numerator, if the null is false, will surely become, given enough data, nonzero as $\sigma_1\neq\sigma_2$ and hence so will be $s_1$ and $s_2$. In the denominator, the $\sigma_j$ will no longer be consistently estimated by the pooled estimator $s^2$. Still, that estimator may be expected to tend to some finite value, say, $s^2\to \tilde{\sigma}^2$. Since we however divide by $n_1$ and $n_2$, $\widehat{S.E.}(s_1-s_2)$ will become very small as the sample size grows, and hence $\hat{Z}$ will become large, so that we will still often correctly reject the null.

Hence, either procedure can be justified on asymptotic grounds. Which is better in finite samples is another matter (that is often adressed via simulation studies).

Here is an illustration (code below) that both variants match the theoretical null distribution (and are close to the $Z$ version which we may compute for illustration when we know the true values in a simulation) under the null.

By playing around with the true values sigma1 and sigma2 you can likewise illustrate that the tests also all have power, i.e., have different distributions than the standard normal if the null is false

enter image description here

Code:

Zhat <- function(x1, x2){
  n1 <- length(x1)
  n2 <- length(x2)
  s1 <- sd(x1)
  s2 <- sd(x2)
  s <- sd(c(x1,x2)) # pooled estimate
  Zhat <- (s1-s2)/sqrt(s1^2/(2*n1)+s2^2/(2*n2))
  Zhat.pooled <- (s1-s2)/sqrt(s^2*(1/(2*n1)+1/(2*n2)))
  Z <- (s1-s2)/sqrt(sigma1^2/(2*n1)+sigma2^2/(2*n2)) # as we know true values in this illustration, we may plug them in, too
  return(list(Zhat, Zhat.pooled, Z))
}

sigma1 <- 1
sigma2 <- 1
n1 <- 40
n2 <- 60

MC.function <- function(sigma1, sigma2, n1, n2){
  x1 <- rnorm(n1, sd=sigma1)
  x2 <- rnorm(n2, sd=sigma2)
  Zhat(x1, x2)
}
MC <- replicate(10000, MC.function(sigma1, sigma2, n1, n2))

plot(density(unlist(MC[1,])), lwd=2, col="blue", main="simulated null distributions", ylim=)
lines(density(unlist(MC[2,])), lwd=2, col="salmon")
lines(density(unlist(MC[3,])), lwd=2, col="green")
x <- seq(-4,4,by=0.1)
lines(x, dnorm(x), lwd=2, col="brown")

P.S.: Basically the same idea is discussed these threads: T-test for Bernoulli Distribution- Sample or Population data for SE calculation? Which variance estimate to use for a Wald test?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.