1
$\begingroup$

Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian distribution so that $P(X_{1}, X_{0}\mid Y=y)\sim \mathcal{N}(\mu_{y},\Sigma)$, with $y\in\{0,1\}$. Notice that the covariance matrix is the same for both values of $Y$. Suppose I also have the distribution of $Y$, but because $Y$ is binary, this distribution can be summarized with one parameter $p:=P(Y=1)$.

Now using Bayes' rule I want to find the conditional distribution of $Y$ given $X_{0}$ and $X_{1}$:

\begin{align} P(Y=1\mid \mathbf{X}=\mathbf{x}) &= {P(\mathbf{X}=\mathbf{x}\mid Y=1)P(Y=1) \over P(\mathbf{X}=\mathbf{x})}\\ &= {p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) \over p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) + (1-p)\exp(-\frac{1}{2}(\mathbf{x}-\mu_{0})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{0}))} \end{align}

Notice that the marginal distribution of $\mathbf{X}$ is a Gaussian mixture. Is there a cleaner expression for this conditional distribution? I have been playing around with it but I don't seem to get anywhere.

$\endgroup$
2
  • 2
    $\begingroup$ i've not seen anything cleaner $\endgroup$ Commented Oct 24, 2022 at 17:42
  • 1
    $\begingroup$ This is very clean indeed, everything in closed form. You can remove the$$\exp\{-\mathbf x^\top\Sigma^{-1}\mathbf x/2\}$$ from all expressions since the covariance matrix is common to both components. $\endgroup$
    – Xi'an
    Commented Oct 25, 2022 at 6:51

1 Answer 1

2
$\begingroup$

$${p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) \over p\exp(-\frac{1}{2}(\mathbf{x}-\mu_{1})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{1})) + (1-p)\exp(-\frac{1}{2}(\mathbf{x}-\mu_{0})^{\top}\Sigma^{-1}(\mathbf{x}-\mu_{0}))}$$ can be simplified into $${p\exp(\mathbf x^\top\Sigma^{-1}\mu_1-\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}) \over p\exp(\mathbf x^\top\Sigma^{-1}\mu_1-\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}) + (1-p)\exp(\mathbf x^\top\Sigma^{-1}\mu_0-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0})}$$ or $$1\Big/1+\frac{1-p}{p}\exp\left(\mathbf x^\top\Sigma^{-1}(\mu_0-\mu_1)-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0}+\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}\right) $$

$\endgroup$
2
  • 1
    $\begingroup$ The second expression is exactly what I wanted! An expression where the covariance matrix appeared in only one exponential. Thank you so much! $\endgroup$
    – Sergio
    Commented Oct 25, 2022 at 7:28
  • $\begingroup$ Depending on your costs, and the setting, like is $\mathbf x$ taking many values versus the parameter being fixed, or the parameters moving and $\mathbf x$ being fixed, some improvement can be gained. For instance,$$\left(\mathbf x^\top\Sigma^{-1}(\mu_0-\mu_1)-\frac{1}{2}\mu_{0}^{\top}\Sigma^{-1}\mu_{0}+\frac{1}{2}\mu_{1}^{\top}\Sigma^{-1}\mu_{1}\right)$$can also be rewritten as$$\left([\mathbf x-\mu_0/2]^\top\Sigma^{-1}\mu_0-[\mathbf x-\mu_1/2]^\top\Sigma^{-1}\mu_1\right)$$only involving two uses of $\Sigma$. $\endgroup$
    – Xi'an
    Commented Oct 25, 2022 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.