How does number of observations supporting alternate hypothesis on a test of a variance have to scale so that null is rejected?

Question

Informal explanation: In the course of my research I've run into the following problem: I am observing a machine that outputs random numbers. Most (if not all) of these random numbers come from the Gaussian distribution $\mathcal{N}(0,\sigma_0^2)$. I know the value of the variance $\sigma_0^2$. However, some proportion of the numbers I observe may come from another distribution $A$. I know that the mean of $A$ is zero, and that $A$ is symmetric (not sure if symmetry helps us). I also know that its variance is $\sigma_a^2>\sigma_0^2$, but I do not know the exact value of $\sigma_a^2$ (unlike $\sigma_0^2$.) The presence of random numbers from $A$ indicates a problem with the machine I am watching. Note that even in that case, my machine will mostly output random values from $\mathcal{N}(0,\sigma_0^2)$.

I would like to construct a hypothesis test on the set $\mathbf{X}$ of $n$ observations I have collected from my machine to determine whether my machine is malfunctioning. Assume that $n$ is large. Null hypothesis is that machine operates normally (i.e. $x_i\sim \mathcal{N}(0,\sigma_0^2)$ where $x_i\in\mathbf{X}$). Alternate hypothesis is that machine is malfunctioning (that is, $x_i \sim \mathcal{N}(0,\sigma_0^2)$ with $x_i\in \mathbf{X}_N$, and $y_i\sim A$ with $y_i\in \mathbf{X}_A$, where $\mathbf{X}_N\cup \mathbf{X}_A=\mathbf{X}$ and $\mathbf{X}_N\cap \mathbf{X}_A=\emptyset$.)

My question is: what is the proportion $\pi$ of observations generated by $A$ in my set of observations $\mathbf{X}$ so that big given enough size of that set $n$, I achieve arbitrarily small type I and type II errors? I.e. how many out of $n$ observations in $\mathbf{X}$ have to come from $A$ for me to determine with negligible error that my machine is broken?

My intuition, based on the CLT, suggests that that if more than $\sqrt{n}$ observations come from $A$, then null hypothesis will be rejected with vanishingly small sum of errors for large $n$. Is my intuition correct? If so, how can I back it up? If not, why? Does it depend on the form of $A$ (what if I told you that $A$ is the sum of $\mathcal{N}(0,\sigma_0^2)$ and another, possibly Gaussian, distribution?)

Attempt at a formal statement of the problem and conjecture: For a zero-mean symmetric mixture distribution $(1-\pi)\mathcal{N}(0,\sigma_0^2)+\pi A(\sigma_a^2)$, I construct a test between two hypotheses:

$$\begin{align} H_0:&\pi=0\\ H_1:&\pi\neq 0 \end{align}$$

with $0\leq\pi\leq1$, $\sigma_a^2>\sigma_0^2$, with $\sigma_0^2$ known. Null hypothesis states that the distribution is a known "pure" Gaussian, while the alternate hypothesis is that it's a Gaussian "contaminated" with $A$.

For that test I need to characterize the value of $\pi$, in terms of $n$, that allows me to separate the two hypotheses. I conjecture that for $\pi>1/\sqrt{n}$ I can reject $H_0$ with arbitrarily small sum of probabilities of type I and type II errors given arbitrarily large (but finite) number of observations $n$. I am ok with $n$ depending on $\sigma_0^2/\sigma_a^2$ -- I assume that those are given to me and do not change as sampling progresses (I think given large enough $n$, whether I can separate the hypothesis depends only on $\pi$, but please correct me if I am wrong.)

I think someone in the statistics community has to have looked at this, and the answer is out there, as this seems like a problem that would come up fairly often. Unfortunately, I've started getting into mathematical statistics very recently: my background is computer science with some graduate-level course work on probability theory, signal theory, and information theory. I would appreciate any help you have to offer as I learn this.

Attempt at solution: I have tried the following track for solving this: suppose I collect $n$ observations in $\mathbf{X}$ and use chi-square test for testing the following hypothesis:

$$\begin{align} H_0:&\sigma^2=\sigma_0^2\\ H_1:&\sigma^2>\sigma_0^2\\ \end{align}$$

If $\sqrt{n}$ observations from $A$ separate the hypotheses for this test, then I've got my proof.

I can compute the statistic for the test (sample variance times $(n-1)/\sigma_0^2$). Given a very large $n$ that I have, it becomes normally-distributed by CLT. Now, let's assume that out of $n$ observations, $\sqrt{n}$ came from $A$ (rest are from $\mathcal{N}(0,\sigma_0^2)$. My test statistic then is a sum of two random variables: the sum of squares of $\sqrt{n}$ values coming from $A$ and the sum of squares of $n-\sqrt{n}$ values coming from $\mathcal{N}(0,\sigma_0^2)$. These two random variables are also (approximately) normally-distributed (very large $n$). Then I think I can look at the performance (i.e. errors) of the test, given the distribution of the test statistic -- I think I need to bound it asymptotically. Unfortunately, the Chernoff and Chebyshev bounds seem to be in the "wrong" direction. That's where I'm stuck.

Am I on the right path? Did I make mistake(s) in my assumptions? Any ideas on how to proceed from here?

Answer 1

People please correct me, but I have the impression that, given the set up, this is a simple problem that does not even require asymptotic theory, because, as it is posed, involves an alternative situation (the "malfunctioning" case) that possibly affects the assumption on the distribution family also (in case the distribution $A$ is not also normal) -and not just the value of some parameter characterizing necessarily the same distribution family. So it is akin to a general misspecification test, and it seems to me that a pure Fisherian approach is the appropriate one here, where the decision to reject or not the null hypothesis is separate from the test results.

Therefore we can specify only the following null hypothesis

$$H_0 : X_i\sim N(0, \sigma^2_0)$$

without an alternative. Under this null-hypothesis, a statistic based on the bias-corrected sample variance $s^2 = \frac 1{n-1}\sum_{i=1}^n(X_i-\bar X)^2$ has a finite-sample chi-square distribution,

$$q_n \equiv(n-1)\frac{s^2}{\sigma^2_0} \sim\Big|_{H_0} \mathcal \chi^2_{n-1}$$

So for whatever sample size $n$, we can calculate the value of the statistic $q_n$, and matching it to the values from the CDF of a chi-square with $n-1$ degrees of freedom, obtain the corresponding $p$-value. The smaller the $p$-value, the lower credibility does the data provide to the null Hypothesis.
Rejection of the null-hypothesis (which, in the Fisherian spirit, has to do with the OP's preferences/guidelines and not with the statistical framework used and the result obtained), signals that probabilistically the machine is malfunctioning, irrespective of the reasons why, or how (I guess deterministic engineering will take over at that point).

A caveat to the suitability of this approach to the problem at hand: It may be the case that "machine is broken" may imply a tolerance threshold for the proportion of random numbers coming not from $N(0, \sigma^2_0)$. I.e. it is not about uncertainty of whether we have contamination or not, but of whether the degree of contamination is above or below a threshold -if it is below a threshold (including the possibility of zero-contamination), then the machine performs adequately and it is not considered "broken". This would require a different approach -but if this is the the true issue here, it is not spelled out in the OP's post, where his attempts to formalize the case indicated a $0/1$ situation.