Is there a standard approach for testing goodness-of-fit to a probability distribution when the samples are independent but not identically distributed? In other words, given the data $\{\{x_1, y_1\}, \{x_2, y_2\},..., \{x_n, y_n\}\}$ and the candidate density function $f(x;y)$, what is an accepted approach for testing whether the $x_i$ are sampled from $X|y_i \sim f(x; y_i)$? What are the assumptions of said approach?


In the comments, whuber suggests it is more appropriate to call $y_i$ a parameter for the distribution on $x_i$ and requests more information about how $f(x; y)$ depends on $y$. NB, I'm interested in "standard methods" more than a custom solution, so the answer should not depend on the details of $f$. That said, in my application, $f$ is a mixture of $m$ Dirichlet-Multinomial distributions (so $x$ is discrete, I asked about continuous variables above in order to reference the K-S test). The vector $\{y_{i,1},y_{i,2},...,y_{i,m}\}$ is the mixture weighting. The parameter for each Dirichlet-Multinomial distribution is independent of $y_i$.

  • 2
    $\begingroup$ You don't quite define a probability distribution: you need also to specify a distribution for the $x_i$. When you do, that will determine a single univariate distribution for $f(x,y)$ which can then be tested. $\endgroup$
    – whuber
    Commented Apr 10, 2014 at 21:45
  • 1
    $\begingroup$ Hmm. I'm not sure the KS is a good 'starting point' for thinking about this problem. I don't currently have a good notion of an especially good way to think about it without specifying the kinds of alternatives you want to be able to identify, though. $\endgroup$
    – Glen_b
    Commented Apr 10, 2014 at 23:23
  • $\begingroup$ @whuber Can you explain further? $f$ is the univariate density for each $x_i$ in my problem statement. What is missing? The joint density on $\{x, y\}$ is unknown, and unnecessary because y is given in the application. $\endgroup$
    – Ian
    Commented Apr 11, 2014 at 13:52
  • $\begingroup$ @Glen_b Thanks for thinking about the question? What other information can I provide to help you come up with a different starting point? $\endgroup$
    – Ian
    Commented Apr 11, 2014 at 13:56
  • $\begingroup$ I think I follow: because you state the $y_i$ are "given" (and presumably measured without error) then, in effect, the $y_i$ play the roles of parameters rather than random variables. (The notation "$X|y_i$" suggests otherwise, unfortunately.) Several methods to deal with this situation come to mind, depending on exactly how the $y_i$ control $f$. Could you provide some details of that? $\endgroup$
    – whuber
    Commented Apr 11, 2014 at 13:57


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.