Upper bound on total variation between two Gaussian mixture

Question

For two random variables $P$ and $Q$ over $R^d$ with distributions $p$ and $q$, respectively, the total variation is defined as

$$ TV(P,Q)=\frac{1}{2}\int_{R^d}\ |p(x)-q(x)|dx. $$

Consider the case where $P$ and $Q$ are Gaussian mixtures each with $2$ components, i.e.,

$$P\sim \pi_1\mathcal{N}(\mu_1,\Sigma_1)+\pi_2\mathcal{N}(\mu_2,\Sigma_2),\\ Q\sim \pi'_1\mathcal{N}(\mu'_1,\Sigma'_1)+\pi'_2\mathcal{N}(\mu'_2,\Sigma'_2)$$.

My question is: can we obtain an upper bound on $TV(P,Q)$ based on the total variations between the Gaussian components, e.g., $TV(\mathcal{N}(\mu_1,\Sigma_1),\mathcal{N}(\mu'_1,\Sigma'_1))$ and $TV(\mathcal{N}(\mu_2,\Sigma_2),\mathcal{N}(\mu'_2,\Sigma'_2))$.

Can triangle inequality ($|a+b|\leq|a|+|b|$) be used here for the choices of $a=\pi_1\mathcal{N}(\mu_1,\Sigma_1)-\pi'_1\mathcal{N}(\mu'_1,\Sigma'_1)$ and $b=\pi_2\mathcal{N}(\mu_2,\Sigma_2)-\pi'_2\mathcal{N}(\mu'_2,\Sigma'_2)$?

Answer 1

Yes - the total variation between measures is a norm, and so in particular, the triangle inequality applies. It is perhaps easiest to think in terms of the triangle inequality applying at the level of densities, to obtain the conclusion at the level of probability distributions.

Now, let $a, b \in [0, 1]$, and let $p_1, p_2, q_1, q_2$ be probability distributions, and compute:

\begin{align} &TV(ap_1 + (1-a) p_2, b q_1 + (1-b)q_2) \\ = &\frac{1}{2}\int_{R^d}\ | \left\{ap_1(x) + (1-a) p_2(x) \right\} - \left\{ b q_1 (x) + (1-b)q_2 (x) \right\} |dx \\ \leqslant &\frac{1}{2}\int_{R^d}\ |ap_1(x) - b q_1 (x)| + |(1-a)p_2 (x) - (1-b)q_2 (x) |dx \\ \end{align}

Now, bound

$$|ap_1(x) - b q_1 (x)| \leqslant a | p_1 (x) - q_1 (x) | + |a - b| q_1 (x)$$

$$|(1-a)p_2 (x) - (1-b)q_2 (x)| \leqslant (1 - a) | p_2 (x) - q_2 (x) | + |a - b| q_2 (x)$$

and integrate, to show that

\begin{align} &TV(ap_1 + (1-a) p_2, b q_1 + (1-b)q_2) \leqslant a \cdot TV(p_1, q_1) + b \cdot TV(p_2, q_2) + |a-b|. \end{align}

Note that this reasoning applies to any mixtures; it is not important that anything is Gaussian.