# Goodness-of-fit Tests

Continuing from my previous question here.

Furthermore, I intend to perform the chi-squared test and plot QQ-plots to test the hypothesis $$H_0:\lambda=1$$. I do not get to see the actual data though; I have only the test result as well as the plot. I want to believe that the data may actually be generated from a geometric distribution with parameter $$p=1−e^{−1}$$, and am worried that the tests I had in mind cannot distinguish between the distributions $$Exp(1)$$ and $$Geom(1−𝑒^{−1})$$, where $$Geom(1−𝑒^{−1})$$ is defined by the pdf $$𝑝(𝑘)=(1−𝑝)^𝑘𝑝, \space \textrm{for} \space 𝑘=0,1,…$$.

Are these two distributions distinguishable if I use the chi-squared test with bins $$[0,1), [1,2), ..., [𝑁−1,𝑁), [𝑁,∞)$$ for some $$𝑁$$?

• It doesn't work to ping someone who hasn't participated in the question yet. Your suggested test won't distinguish exponential from geometric. Practically speaking, however, It's easy to distinguish an exponential from a geometric. If $S=\sum_i X_i$ then $P(S\text{ is an integer}|X_i\sim\text{ exponential}) = 0$, while $P(S\text{ is an integer}|X_i\sim\text{ geometric}) = 1$. So if the sum is exactly an integer, you would bet against it being exponential. Dec 18, 2022 at 11:16
• It depends on where you place the bin boundaries. It's possible to satisfy the condition "all bins contain at least one integer" and still be able to distinguish exponential from geometric. e.g. consider bins "exactly 0", "(0,2)", "exactly 2", "(2,4)", "exactly 4", ... if the odd-numbered bins (first, third, etc) have values in them then the chance it was generated by an exponential is 0. Meanwhile if the even-numbered bins (the second, fourth, etc) typically have higher values in them than the previous bin, it's unlikely to be geometric. Dec 18, 2022 at 12:42
• As sample sizes become very large, you should be able to distinguish almost any two distinct distributions (as long as you don't bin them in such a way as to completely obscure the difference). If you have unbinned data the cdf of the exponential will be smooth, while the geometric will be a step function, and so a Q-Q plot suitable for an exponential will look different. However, if you have the information to generate a Q-Q plot that you can tell apart visually, you would see it in the data values). ... ctd Dec 18, 2022 at 22:28
• ctd... More generally you could distinguish two options by coming up with a classification rule that was best (e.g. in terms of minimizing total expected misclassification error) at distinguishing those two distributions. The QQ plot is useful because it's not limited to only two possibilities; it could show you, for example, when both possible models were wrong (if there's substantial curvature in the QQ plot rather than a difference between a step function and a smooth one but otherwise a straightish relationship); the shape of the curve also tells you something about the distribution.. Dec 18, 2022 at 22:35
• See the two exponentiality plots here (first on simulated and then on real data): stats.stackexchange.com/a/599425/805 ... that curvature in the second one means the values used are clearly not exponential. Dec 18, 2022 at 22:35

If you bin your exponential so that the expected proportions in each bin are exactly the same as you get from a geometric using the same bins, then no, the chi-squared test using those bins has no hope of distinguishing them.

If you don't bin, it's easy to distinguish an exponential from a geometric. If $$S=∑_i X_i$$ then $$P(S\text{ is an integer}|X_i\sim\text{ exponential})=0$$, while $$P(S\text{ is an integer}|X_i\sim\text{ geometric})=1$$. So if the sum is exactly an integer, you would bet against it being exponential, while if the sum is not an integer, you'd know it couldn't be geometric.

If you bin your exponential so that the expected proportions in each bin are not exactly the same as you get from a geometric using the same bins, then eventually you should be able to distinguish them. On that note, in comments you asked the follow-up question:

would these two distributions be distinguishable if I use the chi-squared test with bins that all contain at least one positive integer?

That depends on where you place the bin boundaries. It's possible to satisfy the condition "all bins contain at least one integer" and still be able to distinguish exponential from geometric. e.g. consider bins "exactly 0", "(0,2)", "exactly 2", "(2,4)", "exactly 4", ... if the odd-numbered bins (first, third, etc) have values in them then the chance it was generated by an exponential is 0. Meanwhile if the even-numbered bins (the second, fourth, etc) typically have higher values in them than the previous bin, it's unlikely to be geometric (since with those bins the probabilities should still decrease geometrically).

in this case scenario makes perfect sense to use a QQ-plot as it will be (graphically) possible to tell the distributions apart given a sufficiently large number of samples. Correct me if I'm wrong

As sample sizes become very large, you should be able to distinguish almost any two distinct distributions (as long as you don't bin them in such a way as to completely obscure the difference, naturally).

If you have unbinned data the cdf of the exponential will be smooth, while the geometric will be a step function, and so a Q-Q plot suitable for an exponential will look different. However, if you have the information to generate a Q-Q plot that you can tell apart visually, you would nearly always see it in the data values first).

More generally you could distinguish two options by coming up with a classification rule that was best (e.g. in terms of minimizing total expected misclassification error) at distinguishing those two distributions.

The QQ plot is useful because it's not limited to only two possibilities; it could show you, for example, when both those possible models were wrong (if there's substantial curvature in the QQ plot rather than a difference between a step function and a smooth one but otherwise a straightish relationship); the shape of the curve also tells you something about the distribution.

See the two exponentiality plots here (first on simulated and then on real data): The y-axis has $$\log(x_i)$$, while the x-axis has $$q_i=\log(-\log(1-p_i))$$, where $$p_i$$ here is $$\frac{i-\alpha}{n+1-2\alpha}$$ for a selected $$\alpha$$ between $$0$$ and $$1$$. For large $$n$$ it doesn't matter much which you use; $$0.3$$ and $$\frac{3}{8}$$ are common choices for particular situations. In this case the R default for large $$n$$ of $$\alpha=0.5$$ was used.

For exponential data the trend in these plots should be close to a straight line with a slope of 1. Lines with slope 1 are marked, though a convenient but arbitrary intercept was selected in the case of the second plot.

(The real data comes from here; the original data there was being checked for being Rayleigh, whose squares would then be exponentially distributed, so it's the squares of the original data values that are used as the data in the exponentiality plot here. It's useful to look at more than one simulated-data plot -- they can look distinctly wobbly down at the bottom left corner. I should probably have used the same sample size as the real data, but the underlying point about the way it shows you distributional information should be clear enough I think.)

The clear curvature and strong departure from slope 1 at the low end in the second one means the values used are pretty clearly not exponential; a similar curvature could rule out a geometric. Indeed we can rule out any other Weibull (which would be straight lines with some other slope). In addition we have information about the particular way in which these values are not exponential (they're more peaked at the left end than an exponential) and also not Weibull (they're more right skew than any reasonably-fitting Weibull).

One might well ask why do we take logs and plot quantiles of logs of an exponential rather than do a Q-Q plot of the exponential itself?

That would work; the plot should show constant intercept but varying slope. The main issues as I see it are that the tendency for the 'tail' to wobble about would shift to the high end and be considerably stronger (making the log-scale one easier to read); that we typically want to be able to see the upper tail more precisely; and the other straight line models besides the one-parameter exponential are typically less plausible on the original scale (shifted exponential rather than Weibull). It's often a bit easier to diagnose the deviation in shape on this scale.