0
$\begingroup$

Here is the probability density function (unnormalized) of a covariance matrix: (from a Bayesian perspective):

$$ f(\boldsymbol{V})\propto \det(\boldsymbol{V})^{-\frac{N+J+1}{2}}\int_{\mathbb{R}^{K}}\exp \left [-\frac{1}{2}\sum_{i=1}^{N}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta})^{\top}\boldsymbol{V}^{-1}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta}) \right]d\boldsymbol{\beta} $$

where $\boldsymbol{\beta}$ is a $K \times 1$ vector,

$\boldsymbol{y}_i$ is a $J \times 1$ vector,

$\boldsymbol{X}_i$ is a $J \times K$ matrix,

$\boldsymbol{V}$ is a $J \times J$ covariance matrix, and

$N>K>J$. $\space \mathbb{R}^{K}$ is a $K$-dimensional real space.

The first $J$ columns of $\boldsymbol{X}_i$ are an identity matrix $\boldsymbol{I}_{J \times J}$. This is a multi-dimensional linear regression model.

My question is:

Is the integral $\int_{\mathbb{R}^{K}}\exp \left [-\frac{1}{2}\sum_{i=1}^{N}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta})^{\top}\boldsymbol{V}^{-1}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta}) \right]d\boldsymbol{\beta}$ analytically computable when $\boldsymbol{V}$ is fixed? It seems like a multivariate exponential integral. When $K=J=1$, this is a non-standard univariate inverse chi-squared distribution with $N-1$ degrees of freedom, as proven in Gelman's Bayesian Data Analysis (third edition) pp. 64-66.

$\endgroup$
3
  • $\begingroup$ This is a standard multivariate Normal integral. $\endgroup$
    – whuber
    Commented Sep 13, 2023 at 14:38
  • $\begingroup$ @whuber I tried it, but the fact that the integrated variable is $\boldsymbol{\beta}$ seems to make it difficult. $\endgroup$
    – fan455
    Commented Sep 14, 2023 at 13:49
  • 1
    $\begingroup$ The sum is a sum of quadratic functions of $\beta$ and therefore is a quadratic function of $\beta.$ That means you can express it in the form $-\beta^\prime \Sigma \beta/2 + \tau^\prime \beta + u$ for some square symmetric matrix $\Sigma,$ vector $\tau,$ and number $u.$ In other words, it is a Gaussian function. You know how to integrate those. $\endgroup$
    – whuber
    Commented Sep 14, 2023 at 14:20

1 Answer 1

0
$\begingroup$

Thanks for the hints by @whuber. Let me answer this question myself.

The quadratic term can be written as $$ \sum_{i=1}^{N}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta})^{\top} \boldsymbol{V}^{-1}(\boldsymbol{y}_{i}-\boldsymbol{X}_{i}\boldsymbol{\beta}) =\sum_{i=1}^{N}\boldsymbol{y}_{i}^{\top}\boldsymbol{V}^{-1}\boldsymbol{y}_{i} -2\boldsymbol{\beta}^{\top}\left(\sum_{i=1}^{N}\boldsymbol{X}_{i}^{\top} \boldsymbol{V}^{-1}\boldsymbol{y}_{i}\right) +\boldsymbol{\beta}^{\top}\left(\sum_{i=1}^{N}\boldsymbol{X}_{i}^{\top} \boldsymbol{V}^{-1}\boldsymbol{X}_{i}\right)\boldsymbol{\beta} $$

Let $\boldsymbol{A}=\sum_{i=1}^{N}\boldsymbol{X}_{i}^{\top}\boldsymbol{V}^{-1}\boldsymbol{X}_{i}$, $\boldsymbol{b}=\sum_{i=1}^{N}\boldsymbol{X}_{i}^{\top}\boldsymbol{V}^{-1}\boldsymbol{y}_{i}$, $c=\sum_{i=1}^{N}\boldsymbol{y}_{i}^{\top}\boldsymbol{V}^{-1}\boldsymbol{y}_{i}$. Here, $\boldsymbol{A}$ is a $K \times K$ symmetric positive definite matrix, $\boldsymbol{b}$ is a $K\times 1$ vector, and $c$ is a scalar.

Now the integral term is

$$ \exp\left(-\frac{1}{2}c\right)\int_{\mathbb{R}^{K}} \exp\left(-\frac{1}{2}\boldsymbol{\beta}^{\top}\boldsymbol{A}\boldsymbol{\beta}+ \boldsymbol{b}^{\top}\boldsymbol{\beta}\right)d\boldsymbol{\beta} $$

Applying the Gaussian integral formula (n-dimensional with linear term) from Wikipedia, the integral term is

$$ \exp\left(-\frac{1}{2}c\right) \exp\left(\frac{1}{2}\boldsymbol{b}^{\top}\boldsymbol{A}^{-1}\boldsymbol{b}\right) \sqrt{\frac{(2\pi)^{K}}{\det(\boldsymbol{A})}} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.