About goodness-of-fit p-value and Pearson's $\chi^2$ statistic In a simulation, I generate data from distribution A, and then I hope to test if each of some other distributions B,C,D... can fit the data equally well. Obviously there is an initial fit of the model under distribution A, so I can get the Pearson's chi^2 statistic as a measure of goodness-of-fit. 
My question: is it possible that under distributions B,C,D... I may get a even smaller chi^2 statistic than the one obtained from the true model (i.e. distn A)? The reason for this question is that I am considering whether a 1 or 2-sided test should be done, i.e. H0: model A fits the data; Ha: not H0.
Intuitively (maybe I am wrong), a more general model (I mean, more general than A, such as A=Poisson, B=negative binomial) might provide an equally well or better fit for the data generated from A, so the Pearson's statistic is expected to be smaller if I use distn B to model data simulated from distn A. In this case, I should get a large p-value indicating no lack-of-fit, right? Any suggestions on choosing alternative statistics for evaluating GOF?
 A: I don't see a true connection to one-sided tests.  First let's be sure we understand what a goodness-of-fit does.  We start with a hypothesized distribution.  In the real world there is no exact distribution that the data is generated from.  The distribution is a model. The purpose of the goodness of fit test is to determine whether or not the data suggests that our model is wrong.  So an issue in goodness of fit is how big a departure from the hypothesized model matters for my practical purposes. So the sample size should be chosen to have good power to reject the null hypothesis when the difference is large enough to matter to us.  This is not a very precise notion but I think that when we do data analysis we have a sense of when the data is far from an assumed normal population for example.  We understand this when we look at parameters such as skewness and kurtosis or we see the departutes on a qq plot.
When we use chi-square we do not have an exact test.  So the significance level of the test is only approximate.  This will be a problem when the sample size is small (in addition to possible lack of power).
I think when you seriously consider these facts it should not be surprising that when you artificially sample from a particular distribution and pick a similar (but different distribution) to test a given sample could easily lead to a smaller or larger chi-square value then if the "true" distribution were hypothesized for the fit.  Random variability, estimation of unknown parameters, sample size, approximation of the chi-square test statistic are all factors that enter into this. 
A: I think it depends on what you mean by "different distribution". E.g if you generate data that is $\sim(\mathcal{N}(0,10)$ and compare it to a distribution such as $\sim(\mathcal{N(0.02, 10)}$ then sure, in some cases the second distribution will fit better, just by chance, and will almost certainly fit just about as well. 
A: There are a couple great answers, which addressed most of the constituent questions. I want to address this part of the question "The reason for this question is that I am considering whether a 1 or 2-sided test should be done, i.e. H0: model A fits the data; Ha: not H0."
With Pearson's d goodness of fit tests the null hypothesis is that your dataset could have been generated by a particular distribution. The alternative hypothesis is that it is unlikely to have been generated by the hypothesized distribution. We'd normally refer to such alternative hypotheses as 2 sided. One way to think about why you don't have 1-sided alternative hypothesis for goodness of fit is to think about the fact that the total number of observed counts must equal the total number of expected counts. Because of this fact if the observed counts are higher than the expected counts for some outcomes, to keep the sums equal, the observed counts must be lower than the expected counts for some other outcomes. So only 2-sided alternative hypotheses make sense. 
However, the null hypothesis is rejected if the observed Pearson's d statistic is greater than the critical value of Pearson's d, for the respective degrees of freedom and alpha. We'd normally refer to such decision rules as 1-sided. There is no 2-sided decision rule with Pearson's goodness of fit test. The intuition as to why the decision rules are one sided comes from the formula for Pearson's d statistic. 

Notice that the difference between the observed and expected counts is squared so any directionality is lost. 
Hopefully this helps.
