I am trying to approximate one distribution with another by matching their first two moments. However, due to a discrete (integer) nature of parameters of the approximating distribution, it is not possible to match both moments exactly. Furthermore, the errors are not independent, so that reducing a mismatch on one of the moments, say the mean, might lead to an increase in the error on the other.

I am looking for a some kind of "optimal" constraint on these errors, which would minimise some metric on the difference between the two distributions, say total variation distance. (I put the word optimal in the quotation brackets since its meaning depends on the loss function used to measure the difference between distributions, and I do not want to constrain responses to a specific measure.)

I thought about minimizing a weighted integral of an absolute difference between the first three terms of the expansion of the moment-generating function, i.e. kind of like the total variation distance, but on a truncated expansion of MGFs:

$$ \int_{-\infty}^\infty \left| t\left(\mu_{1,1} - \mu_{2,1}\right) + \frac{t^2}{2!}\left(\mu_{1,2} - \mu_{2,2}\right) \right|g(t)dt\ $$

where $\mu_{i,j}$ is the j-th central moment of the i-th distribution, however I am not certain of the choice of the weighting function $g(t)$, since the integral would not converge without one, and what are the theoretical implications for PDFs of minimising a certain metric on MGFs, e.g. does $d(M_1,M_2) < C$ imply that $d(P_1,P_2) < C$ also, or does $d1(M_1,M_2) < C1$ implies that $d2(P_1,P_2) < C2$ for some metrics $d, d1, d2$.

  • 2
    $\begingroup$ There is a rich, infinite variety of metrics one could construct. How to choose among them depends on why you're doing this. What's the objective of your moment-matching exercise? What is the nature of the data? What is the underlying probability model? Until you edit this post to include such information, it would be difficult to give objective responses to the request for "theoretical implications," either. Out of curiosity, what parametric family with discrete parameters are you using? That seems unusual. $\endgroup$
    – whuber
    May 27, 2015 at 20:08
  • $\begingroup$ Hi, whuber. Thanks for taking time to look at and comment on my post. To answer your questions one at the time: (1) the objective is to approximate one distribution with another; (2) data is integer-valued on [0, N]; (3) the approximating distribution is (standard) hypergeometric; (4) by “theoretical implications” I meant “what does minimising a certain metric on moment-generating functions imply for probability distribution functions?”; (5) not sure what you meant by “what is the underlying probability model” (please forgive my ignorance of the parlance). $\endgroup$
    – Confounded
    May 27, 2015 at 20:24
  • 1
    $\begingroup$ Did you try $\chi^2$? It's popular in physical sciences and would seem to fit your constraints $\endgroup$
    – Aksakal
    May 27, 2015 at 20:26
  • $\begingroup$ Hi, Aksakal. Thank you for commenting on my post. Not sure, however, what you meant by "did you try $\chi^2$?” Are you asking if I have tried using $\chi^2$ as an approximating distribution? I have not (and I might now give it a go that you have mentioned it, thank you), but my question is about something else. $\endgroup$
    – Confounded
    May 27, 2015 at 20:39
  • 2
    $\begingroup$ Matching moments to approximate one distribution by another is a procedure but it's not an objective. Why are you doing this? Given that you're running into limitations, it is natural--and often well worthwhile--to back up and consider whether other procedures might work as well or even better to reach your goals. At this point you haven't made it possible to anyone to help you in this manner because we don't know what your ultimate problem is. $\endgroup$
    – whuber
    May 27, 2015 at 20:43