In a simulation, I generate data from distribution A, and then I hope to test if each of some other distributions B,C,D... can fit the data equally well. Obviously there is an initial fit of the model under distribution A, so I can get the Pearson's chi^2 statistic as a measure of goodness-of-fit.
My question: is it possible that under distributions B,C,D... I may get a even smaller chi^2 statistic than the one obtained from the true model (i.e. distn A)? The reason for this question is that I am considering whether a 1 or 2-sided test should be done, i.e. H0: model A fits the data; Ha: not H0.
Intuitively (maybe I am wrong), a more general model (I mean, more general than A, such as A=Poisson, B=negative binomial) might provide an equally well or better fit for the data generated from A, so the Pearson's statistic is expected to be smaller if I use distn B to model data simulated from distn A. In this case, I should get a large p-value indicating no lack-of-fit, right? Any suggestions on choosing alternative statistics for evaluating GOF?
Thank you very much!