2
$\begingroup$

I want to calculate \begin{align} \int_{-\infty}^{\infty} G(x-v_i, \Sigma_i) G(x-v_j, \Sigma_j) dx \end{align} where \begin{align} G(x-v_i, \Sigma_i) = \frac{1}{(2\pi)^{d/2} |\Sigma_i|^{1/2}} \exp\left\{ -\frac{1}{2} (x - v_i)^T (\Sigma_i)^{-1} (x - v_i) \right\} \end{align}

This is my process \begin{align} &G(x-v_i, \Sigma_i) G(x-v_j, \Sigma_j) \\ &= \frac{1}{(2\pi)^{d/2} |\Sigma_i|^{1/2}} \exp\left\{ -\frac{1}{2} (x - v_i)^T (\Sigma_i)^{-1} (x - v_i) \right\} \frac{1}{(2\pi)^{d/2} |\Sigma_j|^{1/2}} \exp\left\{ -\frac{1}{2} (x - v_j)^T (\Sigma_j)^{-1} (x - v_j) \right\} \\ &= \frac{1}{(2\pi)^{d/2} (2\pi)^{d/2} |\Sigma_i|^{1/2}|\Sigma_j|^{1/2}} \exp\left\{ -\frac{1}{2} (x - v_i)^T (\Sigma_i)^{-1} (x - v_i) -\frac{1}{2} (x - v_j)^T (\Sigma_j)^{-1} (x - v_j) \right\} \\ \end{align} Here, being careful with \begin{align} & \exp\left\{ -\frac{1}{2} (x - v_i)^T (\Sigma_i)^{-1} (x - v_i) -\frac{1}{2} (x - v_j)^T (\Sigma_j)^{-1} (x - v_j) \right\} \\ &= \exp\left\{ -\frac{1}{2} (x - v_i)^T (\Sigma_i)^{-1} (x - v_i) -\frac{1}{2} (x - v_j)^T (\Sigma_j)^{-1} (x - v_j) \right\} \\ &= \exp\left\{\frac{1}{2}( -x^T \Sigma_i^{-1}x + x^T \Sigma_i^{-1}v_i + v_i^T \Sigma_i^{-1}x -v_i^T\Sigma_i^{-1}v_i -x^T \Sigma_j^{-1}x + x^T \Sigma_j^{-1}v_j + v_j^T \Sigma_j^{-1}x -v_j^T\Sigma_j^{-1}v_j ) \right\} \\ &= \exp\left\{\frac{1}{2}( -x^T \Sigma_i^{-1}x + 2x^T \Sigma_i^{-1}v_i - x^T \Sigma_j^{-1}x + 2x^T \Sigma_j^{-1}v_j ) \right\} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i) \right \} \\ &= \exp\left\{\frac{1}{2}( -x^T (\Sigma_i^{-1}+\Sigma_i^{-1})x + 2x^T( \Sigma_i^{-1}v_i + \Sigma_j^{-1}v_j ) ) \right\} \exp\left\{ \frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i) \right\} \\ &= \exp\left\{ \frac{1}{2}( -x^T \Sigma_{ij}^{-1}x + 2x^T\Sigma_{ij}^{-1}v_{ij} - v_{ij}^T\Sigma_{ij}^{-1}v_{ij} ) \right\} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i + v_{ij}^T\Sigma_{ij}^{-1}v_{ij} ) \right\} \\ &= \exp\left\{ -\frac{1}{2} (x - v_{ij})^T \Sigma_{ij}^{-1} (x - v_{ij}) \right\} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i + v_{ij}^T\Sigma_{ij}^{-1}v_{ij} ) \right\} \end{align} where I used the following notation \begin{align} \Sigma_{ij} = (\Sigma_i^{-1} + \Sigma_j^{-1})^{-1} \\ v_{ij} = \Sigma_{ij} \Sigma_i^{-1} v_i + \Sigma_{ij} \Sigma_j^{-1} v_j \end{align} Then I can rewrite \begin{align} &G(x-v_i, \Sigma_i) G(x-v_j, \Sigma_j) \\ &= \frac{1}{(2\pi)^{d/2} (2\pi)^{d/2} |\Sigma_i|^{1/2}|\Sigma_j|^{1/2}} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i + v_{ij}^T\Sigma_{ij}^{-1}v_{ij} )\right\} \nonumber \\ &\times \exp\left\{ -\frac{1}{2} (x - v_{ij})^T \Sigma_{ij}^{-1} (x - v_{ij}) \right\} \end{align} And using this fact \begin{align} \int G(x-v_{ij}, \Sigma_{ij}) = \int \frac{1}{(2\pi)^{d/2} |\Sigma_{ij}|^{1/2}} \exp\left\{ -\frac{1}{2} (x - v_{ij})^T (\Sigma_{ij})^{-1} (x - v_{ij}) \right\} = 1  \\ \int \exp\left\{ -\frac{1}{2} (x - v_{ij})^T (\Sigma_{ij})^{-1} (x - v_{ij}) \right\} = (2\pi)^{d/2} |\Sigma_{ij}|^{1/2} \end{align} Then \begin{align} \int G(x-v_i, \Sigma_i) G(x-v_j, \Sigma_j) &= \frac{(2\pi)^{d/2} |\Sigma_{ij}|^{1/2}}{(2\pi)^{d/2} (2\pi)^{d/2} |\Sigma_i|^{1/2}|\Sigma_j|^{1/2}} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i + v_{ij}^T\Sigma_{ij}^{-1}v_{ij} )\right\} \\ &= \frac{ |\Sigma_{ij}|^{1/2}}{(2\pi)^{d/2} |\Sigma_i|^{1/2}|\Sigma_j|^{1/2}} \exp\left\{\frac{1}{2}( -v_j^T\Sigma_j^{-1}v_j - v_i^T\Sigma_i^{-1}v_i + v_{ij}^T\Sigma_{ij}^{-1}v_{ij} )\right\}   \\ &= \frac{ |\Sigma_{ij}|^{1/2}}{(2\pi)^{d/2} |\Sigma_i\Sigma_j|^{1/2}} \exp\left\{-\frac{1}{2}v_j^T\Sigma_j^{-1}v_j - \frac{1}{2} v_i^T\Sigma_i^{-1}v_i + \frac{1}{2} v_{ij}^T\Sigma_{ij}^{-1}v_{ij} \right\} \end{align}

However, according to 8.1.8,the answer is \begin{align*} G(v_i-v_j, \Sigma_i+\Sigma_j) \\ = \frac{1}{(2\pi)^{d/2} |\Sigma_i+\Sigma_j|^{1/2}} \exp\left\{ -\frac{1}{2} (v_i - v_j)^T (\Sigma_i+\Sigma_j)^{-1} (v_i - v_j) \right\} \end{align*}

Where did I make a mistake?

$\endgroup$
6
  • $\begingroup$ I didn't go through the whole thing but based on the 8.1.8 answer, it looks like you are calculating the joint pdf of $X_{i} - X_{j}$ where the $X_{i}$ and $X_{j}$ are multivariate normal. .Assuming the two are independent then the means subtract and the variances add so the answer looks correct, assuming the question is what I said. The initial integral you wrote may have problems because it should be some sort of a variant of the convolution of $X_{i}$ and $X_{j}$ and what you have there is not the convolution. This may be why you don't get the same answer. $\endgroup$ – mlofton Sep 5 '18 at 3:07
  • $\begingroup$ sorry. I cannot understand what you said. I want to calculate the first integral. 8.1.8 does not have an integral. so co efficient of 8.1.8 will be what I want. Is this wrong? $\endgroup$ – yoyo Sep 5 '18 at 3:13
  • $\begingroup$ 8.1.8 says that N(m_1, sigma_1) x N(m_2, sigma_2) = c_3 x N(m_3, sigma_3). therefore, taking integral in both sides, what I want is c_3? $\endgroup$ – yoyo Sep 5 '18 at 3:20
  • $\begingroup$ Judging from the length and complexity of this post, you are working too hard. The crux of the idea is that (after removing any normalizing factors) the product of PDFs is computed by summing the arguments of the exponentials. That comes down to the sum of two quadratic forms, which has to be re-expressed algebraically as a single quadratic form. The answer will depend on the determinant of that form. So, drop the integrals, drop the exponentials, work with the forms themselves, and focus on obtaining the determinant. $\endgroup$ – whuber Sep 5 '18 at 12:07
  • $\begingroup$ @Japanese Student: I'm sorry but I don't follow. Usually, (AFAIK ) if you multiply 2 independent densities of $X_1$ and $X_2$ respectively, then the result is the joint density of $X_1$ and $X_2$. The answer, acccording to 8.1.8, assuming G denotes the normal, seems to denote the density of the subtraction of 2 normals. So, I'm not sure what's going on because I'm not sure what the original question is. Is the exact question in the text ? $\endgroup$ – mlofton Sep 6 '18 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.