We want to know if 100 integer values (in a vector
X) are following a Poisson $P(\lambda=2)$ distribution, which is our $H_0$ hypothesis.
Let's say the observed value of the $\chi^2$ with 5 degrees of freedom is $C=7.5$.
With an accepted risk of $\alpha = 20\%$, the threshold for $\chi^2_5$ is $7.29 < C$. Thus we reject the hypothesis $H_0$. Conclusion: we don't know anything about the distribution of the vector $X$, except that it's probably not a Poisson of param. 2.
With an accepted risk of $\alpha = 1\%$, the threshold for $\chi^2_5$ is $15.09 > C$. Thus we don't reject the hypothesis $H_0$. Conclusion: we now know that a good candidate for the distribution of $X$ is a Poisson distribution of parameter $2$.
How is it possible, in this $\chi^2$-goodness-of-fit setting, to:
- not know anything when we accept a high risk
- have a more precise idea when we decide to take a much lower risk!
This seems contradictory with the intuition: if we want a very low risk (1%), we should not be able to draw any conclusion at all here... And if we accept a higher risk (20%), we should be able to draw a conclusion.
Why is it the contrary here?
Note: I have read many questions/answers about Fisher / Neyman-Pearson, such as When to use Fisher and Neyman-Pearson framework? and others etc. but I think it would be useful to understand this precise example before being able to understand the whole theoretical setting.