2
$\begingroup$

I am trying to define a 2 dimensional Gaussian in which each dimension has a different variance.

One way I thought of was just to define it as the product of 1D Gaussians, but I wasn't sure if that would hold up. Numerically plotting the product of two zero-mean, unit-variance distributions appears to give the concentric circles that one would expect in the 2D case. Furthermore, adjusting the variance of one warps the plot to concentric ellipses. However, I want to be able to express it mathematically, not numerically. Mathematically expressed, the product of two 1D Gaussians doesn't match up with the expression I'm used to seeing for a multivariable Gaussian distribution.

Multivariate Gaussian where $x \in \mathbb{R}^2$ :

$$\frac{1}{\sqrt{(2\pi)^{k}|\Sigma|}} \exp \left(-\frac{1}{2}(x - \mu)^{T}\Sigma^{-1}(x - \mu) \right)$$

Product of two 1D Gaussians:

$$\frac{1}{\sigma_{1}\sigma_{2}2\pi} \exp \left( -\frac{(x_1 - \mu_1)^2}{2\sigma_1^2} -\frac{(x_2 - \mu_2)^2}{2\sigma_2^2} \right)$$

Where am I going wrong here? Would setting $\Sigma$ in the multivariate gaussian to diag($\sigma_1$, $\sigma_2$) do the trick?

$\endgroup$
  • $\begingroup$ Your expression for the multivariate gaussian is incorrect: on either side of $\Sigma^{-1}$ you should see the vectors $(x_1-\mu_1, x_2-\mu_2)$. When you make that change it should be obvious how to express $\Sigma$ in terms of $\sigma_1$ and $\sigma_2$. $\endgroup$ – whuber Feb 24 '16 at 20:16
  • $\begingroup$ If I understand your questions right, then you are trying to have the math line up so that the product of 2 independent Gaussians is a bivariate Gaussian. This is not true. math.stackexchange.com/questions/101062/… $\endgroup$ – Greenparker Feb 24 '16 at 20:21
  • 1
    $\begingroup$ Your expression for the 2-variate normal is still wrong. The term after $\Sigma^{-1}$ should also be $(x_1 - \mu_1)$. There is no need for $x_2$ and $\mu_2$. Both $x_1$ and $\mu_1$ are vectors. $\endgroup$ – Greenparker Feb 24 '16 at 20:25
  • 1
    $\begingroup$ @Greenparker I think you may be confusing the issue by conflating the product of random variables (which is not of concern in this question) with the product of pdfs. $\endgroup$ – whuber Feb 24 '16 at 20:29
  • 1
    $\begingroup$ Ah, I see now. Thanks for clarifying @whuber. You had the right idea at the end. I have added a solution. $\endgroup$ – Greenparker Feb 24 '16 at 20:43
7
$\begingroup$

Let $x \sim N_2(\mu, \Sigma)$ where $\Sigma = \left [\begin{array}{cc} \sigma_1^2 & 0\\ 0 & \sigma^2 \end{array} \right]$. Then det($\Sigma)$ = $\sigma_1^2 \sigma_2^2$

\begin{align*} & \dfrac{1}{\sqrt{(2\pi)^2 |\Sigma|}} \exp\left \{ -\dfrac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu) \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[x_1 - \mu_1\quad x_2 - \mu_2 \right]\left [\begin{array}{cc} 1/\sigma_1^2 & 0\\ 0 & 1/\sigma^2 \end{array} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[\dfrac{x_1 - \mu_1}{\sigma^2_1} \quad \dfrac{x_2 - \mu_2}{\sigma^2_2} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = &\dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{ -\dfrac{(x_1 - \mu_1)^2}{2\sigma_1^2} - \dfrac{(x_2 - \mu_2)^2}{2 \sigma_2^2} \right \}. \end{align*}

$\endgroup$
  • $\begingroup$ Ah! It's the determinant being the product that threw me. Thanks! $\endgroup$ – marcman Feb 24 '16 at 20:44
  • 1
    $\begingroup$ +1. But the first line of the formula may need a little fixing up (in the denominator) to match the correct general formula. The algebra goes $$\sqrt{(2\pi)^k|\Sigma|} = \sqrt{(2\pi)^2 \sigma_1^2\sigma_2^2} = 2\pi\sigma_1\sigma_2$$ (assuming the $\sigma_i$ are positive). $\endgroup$ – whuber Feb 24 '16 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.