# Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians

Suppose I have three variables $$X_{1}, X_{2}$$ and $$Y$$, where $$X_{1}, X_{2}$$ are continuous and $$Y$$ is binary. The conditional distribution of $$X_{1}, X_{2}$$, given $$Y$$ is a multivariate Gaussian distribution so that $$P(X_{1}, X_{0}\mid Y=y)\sim \mathcal{N}(\mu_{y},\Sigma)$$, with $$y\in\{0,1\}$$. Notice that the covariance matrix is the same for both values of $$Y$$. Suppose I also have the distribution of $$Y$$, but because $$Y$$ is binary, this distribution can be summarized with one parameter $$p:=P(Y=1)$$.

Now using Bayes' rule I want to find the conditional distribution of $$Y$$ given $$X_{0}$$ and $$X_{1}$$:

\begin{align} P(Y=1\mid \mathbf{X}=\mathbf{x}) &= {P(\mathbf{X}=\mathbf{x}\mid Y=1)P(Y=1) \over P(\mathbf{X}=\mathbf{x})}\\ &= {p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) \over p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) + (1-p)\exp(-\frac{1}{2}(\mathbf{x}-\mu_{0})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{0}))} \end{align}

Notice that the marginal distribution of $$\mathbf{X}$$ is a Gaussian mixture. Is there a cleaner expression for this conditional distribution? I have been playing around with it but I don't seem to get anywhere.

• i've not seen anything cleaner Oct 24, 2022 at 17:42
• This is very clean indeed, everything in closed form. You can remove the$$\exp\{-\mathbf x^\top\Sigma^{-1}\mathbf x/2\}$$ from all expressions since the covariance matrix is common to both components. Oct 25, 2022 at 6:51

$${p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) \over p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) + (1-p)\exp(-\frac{1}{2}(\mathbf{x}-\mu_{0})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{0}))}$$ can be simplified into $${p\exp(\mathbf x^\top\Sigma^{-1}\mu_1-\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}) \over p\exp(\mathbf x^\top\Sigma^{-1}\mu_1-\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}) + (1-p)\exp(\mathbf x^\top\Sigma^{-1}\mu_0-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0})}$$ or $$1\Big/1+\frac{1-p}{p}\exp\left(\mathbf x^\top\Sigma^{-1}(\mu_0-\mu_1)-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0}+\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}\right)$$
• Depending on your costs, and the setting, like is $\mathbf x$ taking many values versus the parameter being fixed, or the parameters moving and $\mathbf x$ being fixed, some improvement can be gained. For instance,$$\left(\mathbf x^\top\Sigma^{-1}(\mu_0-\mu_1)-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0}+\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}\right)$$can also be rewritten as$$\left([\mathbf x-\mu_0/2]^\top\Sigma^{-1}\mu_0-[\mathbf x-\mu_1/2]^\top\Sigma^{-1}\mu_1\right)$$only involving two uses of $\Sigma$. Oct 25, 2022 at 9:37