Closeness of 2-parametric discrete distributions when first 2 moments are matching

Question

Let $\mathcal{D}$ be a particular 2-parameter uni-variate discrete distribution family, and let $D(\theta_1, \theta_2) \in \mathcal{D}$ be one particular distribution from this family, where $\theta_i \in \mathbb{R}$. The mean and variance of this distribution, namely $\mu_D$ and $\sigma_D^2$, are complicated functions (the functions are known) of $\theta_1$ and $\theta_2$, i.e. $\mu_D = f_1(\theta_1, \theta_2)$ and $\sigma^2_D = f_2(\theta_1, \theta_2)$.

I now have a distribution $D' \in \mathcal{D}$ whose parameters are unknown to me. However, I know that the mean and variance of $D'$, namely $\mu_{D'}$ and $\sigma_{D'}^2$, are the same as that of $D$, i.e. $\mu_{D'} = \mu_D$ and $\sigma_{D'}^2 = \sigma_{D}^2$. Although $f_1$ and $f_2$ are known functions, they are hard to invert, which means that it is difficult to find the underlying parameters using $\mu_{D'}$ and $\sigma_{D'}^2$ and trying to invert under $f_1$ and $f_2$.

So, now, we have $D'$ whose first and second moments match $D$. Can I conclude that $D$ and $D'$ are close in total variation distance? Are there any sufficient conditions on the nature of the distributions that will ensure that they are close in total variation distance? Necessary conditions?

===============

My intuition says that they should be the same, but I am far from confident.

Below are what I tried:

1. I know that the moment generating function (MGF) of a distribution uniquely determines the distribution. So I tried to prove that the MGF of $\mathcal{D}$, which is a function of $\theta_1$ and $\theta_2$, can be written as a function of $f_1(\theta_1, \theta_2)$ and $f_2(\theta_1, \theta_2)$. This would complete the proof. However this has proved difficult.

2. I tried deriving an expression for the total variation distance between two different distributions from $\mathcal{D}$ in terms of their first and second moments, but to no avail.

Shall be grateful for any insight whatsoever.

Answer 1

The total variation between two distributions with matching first and second moments can be arbitrarily large. Consider the following two distributions:

1. A normal distribution with mean $\mu_0=0$ and variance $\sigma_0^2$
2. A mixture of normal distributions having weights $p_1=p_2=\frac{1}{2}$, means $\mu_1=-\mu_2=\frac{\sigma_0}{\sqrt2}$, and variances $\sigma_1=\sigma_2=\frac{\sigma_0}{\sqrt2}$

Some observations:

1. Both distributions clearly have mean $=0$, so their first moments will always match
2. Using the formula given here, we see that the second distribution has variance: $\mu_1^2 + \sigma_1^2 = \frac{\sigma_0^2}{2} + \frac{\sigma_0^2}{2} = \sigma_0^2$ and therefore their seconds moments also match

The difference between the distributions at the point $x=0$ is given by: $ \bigg|\ \frac{1}{\sqrt{2\pi\sigma_0^2}} - \Big( \frac{1}{2\sqrt{2\pi\sigma_1^2}} e^{-\frac{\mu_1^2}{2\sigma_1^2}} + \frac{1}{2\sqrt{2\pi\sigma_2^2}} e^{-\frac{\mu_2^2}{2\sigma_2^2}} \Big) \bigg|=$

$ \bigg|\ \frac{1}{\sqrt{2\pi\sigma_0^2}} - \Big( \frac{1}{2\sqrt{\pi\sigma_0^2}} e^{-\frac{1}{2}} + \frac{1}{2\sqrt{\pi\sigma_0^2}} e^{-\frac{1}{2}} \Big) \bigg|=$

$ \bigg|\ \frac{1}{\sqrt{2\pi\sigma_0^2}} - \frac{1}{\sqrt{\pi\sigma_0^2}} e^{-\frac{1}{2}} \bigg|= \frac{1}{\sqrt{\pi\sigma_0^2}} \bigg| \frac{1}{\sqrt{2}} - e^{-\frac{1}{2}} \bigg| \approx \frac{1}{\sqrt{\pi\sigma_0^2}}\cdot 0.1$

As $\sigma_0 \to 0$ this quantity grows without bound, and therefore so does the total variation between the two distributions.

For some intuition about what's going on here, as $\sigma_0 \to 0$ the first distribution is getting narrower around zero more quickly than the means of the two components of the Gaussian mixture can approach it, and therefore the difference between the two distributions at this point blows up.

EDIT

I didn't read the question carefully enough, and thought this was asking more broadly about what statements one could make about arbitrary distributions given their first and second moments.

If we consider our family of distributions $\mathcal{D}$ to be the set of all two-component Gaussian mixtures which are symmetric about the origin and have $\sigma_1=\sigma_2$ and $p_1=p_2=\frac{1}{2}$ (note that this also encompasses univariate Gaussians having mean $=0$ as a degenerate case), then the above example constitutes a counterexample to your conjecture, as all distributions in this family can be uniquely characterized by two parameters:

1. the mean of the non-negative component; this was $0$ for the first distribution, and $\mu_1=\frac{\sigma_0}{\sqrt{2}}$ for the second distribution
2. the standard deviation of the non-negative component; this was $\sigma_0$ for the first distribution, and $\sigma_1=\frac{\sigma_0}{\sqrt{2}}$ for the second distribution

Answer 2

If the parameters each appear in at least one of the terms for the two moments (and each moment involves at least one of them), then aside from degenerate cases this would suffice to identify the parameters (you have two equations in two unknowns -- if a unique solution exists for that then you have identified the distribution, though you can be in a situation where there's more than one solution).

(This only works because you're restricting consideration to a family indexed by two parameters, in general matching moments doesn't pin distributions down)