A simulation gave me the following density of data (grey histogram, blue kernel density estimation). Having an alternative input $I^*$ with the same distribution (green line), I would like to check whether $I^*$ belongs to my simulated data (hypothesis testing).

enter image description here

Is it correct that I have to look at the quantiles of my approximately normal distributed data in order to verify my hypothesis? Say for $\alpha = 0.05$ I calculate the $\alpha$-quantiles and if $I^*$ is located in this area, I reject the hypothesis?

I'm a stranger to hypothesis-testing and appreciate any help. Thank you!


There's a couple of interesting things going one here I want to address:

  • Why are you using simulated data? It appears from your simulation that you probably know the distribution. It looks liked a centered normal with SD of about .15. IF you know your distribution, there is no need to rely on the quantiles of your simulation, you can use the known quantiles of a normal distribution. That being said, quantiles in a simulation approximate quantiles in a distribution when the number of simulations in large, so your general intuition about the alpha level and it's relationship to quantiles is correct.
  • More importantly though, hypothesis tests are not performed on a single data point. A hypothesis test compares whether a set of data (n>1) follows a hypothesized distribution. We can't really make an inferential statements about a single data point. All we can say is "it is in the 96th percentile of the simulated distribution." You can't quantify the confidence of that statement, because you don't have any degrees of freedom to do that.

So in generally, I would say your intuition is correct, but the idea of performing a hypothesis test on a single point is flawed. If you give a little more detail about how you came to this type of analysis and what your trying to accomplish, we may be able to give you a bit more help.

| cite | improve this answer | |
  • $\begingroup$ Thank you for your fast reply! $\endgroup$ – nomeal Apr 10 at 18:28
  • $\begingroup$ I'm using simulated data, because the aim is to visualize some theoretical laws of convergence. But maybe some background: The data is created out of a specifal central limit theorem $ n^{-1/2} (X_k - \mathbb{E}X_k) \rightarrow \mathcal{N}(0,\sigma^2) $. Given that I use the continuuos mapping theorem with $h: \mathbb{R}^k \rightarrow \mathbb{R} $. Therefore my random variables converge under $h$ to a normal distribution as well. Next I approach $\mathbb{E}X_k$ with a lot of simulations and then calculate $h(n^{-1/2} (X_k - \mathbb{E}X_k)) \rightarrow h(\mathcal{N}(0,\sigma^2)) $ $\endgroup$ – nomeal Apr 10 at 18:39
  • $\begingroup$ Now $I^*$ from above is one extra calculation $h(n^{-1/2} (\hat{X}_k - \mathbb{E}X_k))$. My aim is to show with $I^*$, whether I accept or reject $H_0 : X_k \text{ is poisson data} $. $\endgroup$ – nomeal Apr 10 at 18:44
  • $\begingroup$ So to answer your second argument directly: Which seems like a single data point, is in fact $k$ data points, before beeing mapped by $h$. Do you understand my point? $\endgroup$ – nomeal Apr 10 at 18:48
  • $\begingroup$ I think i get the general idea, though I'm not an expert in that area. I don't think I have much more to add. $\endgroup$ – Tanner Phillips Apr 10 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.