1
$\begingroup$

I have a general question as to which methods are considered "standard" or "best practice" in asssessing the following type of hypothesis.

I am running simulations where I generate random graphs according to some prescription. I am interested in a certain graph property which is just an integer, call it $p$.

The one parameter in my simulations is the expected graph size $N$. I run simulations for a range of values of $N$, say $N=2^k$ for $k=1,2,\ldots20$. For each value of $N$ I do, say, 100 runs, obtaining 100 values for $p$.

I have a hypothesis about the distribution of $p$: I believe $p\sim Poisson(\alpha \sqrt N)$, for some $\alpha\in\mathbb R$.

Now I want to test that hypothesis.

I've done a number of things (maximum-likelihood estimation of $\lambda$ for each individual dataset against $Poisson(\lambda)$ and then a visual check how $\lambda$ goes with $N$, maximum-likelihood estimation of the constant $\alpha$ assuming the relationship given above, each one of these followed by different distribution fit test...) but I would be interested to know whether there is a "canonical" or "best practice" approach to this. What would a reader like to see as evidence for the hypothesis? What numbers or plots would be considered appropriate?

$\endgroup$
1
$\begingroup$

Unless you don't have a theoretical model for the unknown parameter $p$ as it seems so, since you are making simulations to obtain its distribution, probably the best hypothesis testing scheme for this case is the two sided composite hypothesis test where you assume that the null hypothesis gives you the true parameter $\alpha_t$ and the alternative gives you any value that is not $\alpha_t$. For this type of test usually the uniformly most powerful test does not exist and the most common way is to consider the GLRT, Generalized maximum likelihood ratio test. There is a solid theory about it such as it is asymptotically uniformly most powerful test, for example. As a result, as the number of observations increases, the detector performance will increase. Whenever a new observation is made the GLRT will output a new decision, $0$ or $1$. The distribution of these zeros and ones over time $t$, the observation index, must match with the theoretically expected one. Then, one can claim that the observations really follow a Poission distribution with parameter $\alpha$.

$\endgroup$
1
  • $\begingroup$ Thank you. I will look into that. Regarding the question in your first sentence: my hypothesis on the distribution of $p$ comes from my theoretical model (if I am using the word in the right sense). What I mean is, given the known distributions of certain other graph parameters, I can do an analytic calculation which leads to the conclusion that $p$ should have the hypothesized distribution, but that calculation involves approximations and hand-wavy arguments, so I want to test it. $\endgroup$ – matimo2 Jan 20 '15 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.