5
$\begingroup$

I am trying to define a 2 dimensional Gaussian in which each dimension has a different variance.

One way I thought of was just to define it as the product of 1D Gaussians, but I wasn't sure if that would hold up. Numerically plotting the product of two zero-mean, unit-variance distributions appears to give the concentric circles that one would expect in the 2D case. Furthermore, adjusting the variance of one warps the plot to concentric ellipses. However, I want to be able to express it mathematically, not numerically. Mathematically expressed, the product of two 1D Gaussians doesn't match up with the expression I'm used to seeing for a multivariable Gaussian distribution.

Multivariate Gaussian where $x \in \mathbb{R}^2$ :

$$\frac{1}{\sqrt{(2\pi)^{k}|\Sigma|}} \exp \left(-\frac{1}{2}(x - \mu)^{T}\Sigma^{-1}(x - \mu) \right)$$

Product of two 1D Gaussians:

$$\frac{1}{\sigma_{1}\sigma_{2}2\pi} \exp \left( -\frac{(x_1 - \mu_1)^2}{2\sigma_1^2} -\frac{(x_2 - \mu_2)^2}{2\sigma_2^2} \right)$$

Where am I going wrong here? Would setting $\Sigma$ in the multivariate gaussian to diag($\sigma_1$, $\sigma_2$) do the trick?

$\endgroup$
8
  • $\begingroup$ Your expression for the multivariate gaussian is incorrect: on either side of $\Sigma^{-1}$ you should see the vectors $(x_1-\mu_1, x_2-\mu_2)$. When you make that change it should be obvious how to express $\Sigma$ in terms of $\sigma_1$ and $\sigma_2$. $\endgroup$
    – whuber
    Commented Feb 24, 2016 at 20:16
  • $\begingroup$ If I understand your questions right, then you are trying to have the math line up so that the product of 2 independent Gaussians is a bivariate Gaussian. This is not true. math.stackexchange.com/questions/101062/… $\endgroup$ Commented Feb 24, 2016 at 20:21
  • 1
    $\begingroup$ Your expression for the 2-variate normal is still wrong. The term after $\Sigma^{-1}$ should also be $(x_1 - \mu_1)$. There is no need for $x_2$ and $\mu_2$. Both $x_1$ and $\mu_1$ are vectors. $\endgroup$ Commented Feb 24, 2016 at 20:25
  • 2
    $\begingroup$ @Greenparker I think you may be confusing the issue by conflating the product of random variables (which is not of concern in this question) with the product of pdfs. $\endgroup$
    – whuber
    Commented Feb 24, 2016 at 20:29
  • 1
    $\begingroup$ Ah, I see now. Thanks for clarifying @whuber. You had the right idea at the end. I have added a solution. $\endgroup$ Commented Feb 24, 2016 at 20:43

1 Answer 1

10
$\begingroup$

Let $x \sim N_2(\mu, \Sigma)$ where $\Sigma = \left [\begin{array}{cc} \sigma_1^2 & 0\\ 0 & \sigma_2^2 \end{array} \right]$. Then det($\Sigma)$ = $\sigma_1^2 \sigma_2^2$

\begin{align*} & \dfrac{1}{\sqrt{(2\pi)^2 \mathrm{det(}\Sigma)}} \exp\left \{ -\dfrac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu) \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[x_1 - \mu_1\quad x_2 - \mu_2 \right]\left [\begin{array}{cc} 1/\sigma_1^2 & 0\\ 0 & 1/\sigma_2^2 \end{array} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[\dfrac{x_1 - \mu_1}{\sigma^2_1} \quad \dfrac{x_2 - \mu_2}{\sigma^2_2} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = &\dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{ -\dfrac{(x_1 - \mu_1)^2}{2\sigma_1^2} - \dfrac{(x_2 - \mu_2)^2}{2 \sigma_2^2} \right \}. \end{align*}

$\endgroup$
2
  • $\begingroup$ Ah! It's the determinant being the product that threw me. Thanks! $\endgroup$
    – marcman
    Commented Feb 24, 2016 at 20:44
  • 1
    $\begingroup$ +1. But the first line of the formula may need a little fixing up (in the denominator) to match the correct general formula. The algebra goes $$\sqrt{(2\pi)^k|\Sigma|} = \sqrt{(2\pi)^2 \sigma_1^2\sigma_2^2} = 2\pi\sigma_1\sigma_2$$ (assuming the $\sigma_i$ are positive). $\endgroup$
    – whuber
    Commented Feb 24, 2016 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.