3
$\begingroup$

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,R\, \text{diag}(S)$, where $S$ is $k\times 1$ vector of standard deviations, $\text{diag}(S)$ is the diagonal matrix with diagonal elements $S$, and $R$ is $k \times k$ correlation matrix. Elements of the covariance matrix $\Sigma$ can be written as follows:

  • $\Sigma_{ii} = s^{2}_{ii}$
  • $\Sigma_{ij} = s_i s_j r_{ij}$.

Now let's assume that $\Sigma$ has a stadard inverse-Wishart distribution $W^{-1}_k(I,\nu), \nu\geq k$,that is, when $\Sigma$ has density function $$f_k(\Sigma;I,\nu) \propto |\Sigma|^{-\frac{1}{2}(\nu + k+1)}\, \exp(-\frac{1}{2}\operatorname{tr}(\Sigma^{-1}))$$ Now under the transformation $\Sigma \rightarrow (S,R)$ the Jacobian has value of $2^k\left(\displaystyle \prod _{i}^k s_i\right)^{k}$. The density function under this transformation is transformed into the following form $$f_k(S,R;I,\nu) \propto |R|^{-\frac{1}{2}(\nu + k+1)} \displaystyle \prod_i^k s_i^{-(\nu+1)} \exp\left(-\frac{r^{ii}}{2 s_i^2}\right) $$ where $r^{ii}$ is the $i$th diagonal element of $R^{-1}$.

Can someone point me in the right direction to compute this Jacobian?

$\endgroup$
3
$\begingroup$

The algebra of differential forms makes short work of this.

I take it that $\Sigma = (\sigma_{ij})$ is coordinatized by means of one of its triangles, say the upper triangle with components $\sigma_{ij},$ $1\le i\le j \le k.$ The differential element of those components, $|\prod_{ij}\mathrm{d}\sigma_{ij}|,$ therefore is the (absolute) wedge product of these $\mathrm{d}\sigma_{ij}.$ Using

$$\mathrm{d} \sigma_{ii} = 2s_i \mathrm{d}s_i\tag{1}$$

and

$$\mathrm{d} \sigma_{ij} = s_i s_j \mathrm{d} r_{ij} + s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i;\ i \ne j\tag{2}$$

it's surprisingly easy to simplify the differential element in the new coordinates:

$$\eqalign{ \prod_{ij}\mathrm{d}\sigma_{ij} &= \pm\prod_i \mathrm{d}\sigma_{ii} \wedge \prod_{i\lt j} \mathrm{d}\sigma_{ij} \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i + s_is_j\mathrm{d}r_{ij}\right) \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_is_j\mathrm{d}r_{ij}\right) \\ &= \left(\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j\right)\ \mathrm{d} s_1\wedge \cdots \wedge \mathrm{d}s_k\ \wedge\ \mathrm{d} r_{12} \wedge \cdots \wedge dr_{k-1,k}. }$$

The second line substituted the differentials $(1)$ and $(2)$ determined by the transformation. The third line recognized that any term involving $\mathrm{d}s_i$ in the right hand product would be multiplied by one of the $\mathrm{d}s_i$ in the left hand product, resulting in zero because $\mathrm{d}s_i\wedge \mathrm{d}s_i=0.$ The last line merely pulls the constants (the "zero-forms") over to the left.

By definition, the Jacobian is the absolute value of those constants. Evidently each $s_i$ appears $k$ times. The absolute value is determined from the fact that all the $s_i$ are non-negative, yielding

$$|\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j| = 2^k \left(\prod_i s_i\right)^k,$$

QED.


To justify the use of this Jacobian in the Wishart context, we also need to know the transformation between $\Sigma$ and $(S,R)$ is almost surely one-to-one. But that's clear, because $(S,R)$ obviously determines $\Sigma;$ while the diagonal of $\Sigma$ determines the $s_i$ which then can be used to compute the $r_{ij}$ from the $\sigma_{ij}$ when $i\ne j.$

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

At first I was a little bit puzzled how to transform covariance matrix the way I would transform random variables (e.g. $Y_1 = X_1X_2$). Then I realized that covariance matrix $\Sigma = (\sigma_{ij})$ of order $k$ consists of $k^2$ variables and only of $\frac{k(k+1)}{2}$ unique variables (i.e. $\sigma_{ij} = \sigma_{ji}$), which can be divided into two groups (diagonal elements and let's say upper triangle without the diagonal)

  1. $\sigma_{ii} = s_{ii}^2$
  2. $\sigma_{ij} = s_is_jr_{ij}$

So I have $\frac{k(k+1)}{2}$-dimensional random variable $$\mathbf{X} = (\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k},\sigma_{11},\ldots,\sigma_{kk})$$ and look for $\frac{k(k+1)}{2}$-dimensional random variable $$\mathbf{Y} = (r_{12},\ldots,r_{1k},r_{23},\ldots,r_{2k},\ldots,r_{(k-1)k},s_{11},\ldots,s_{kk})\text{,}$$ where the Jacobian of the transformation has the following form

\begin{eqnarray*} \mathbf{J} &=& \begin{vmatrix} \frac{\partial \sigma_{12}}{\partial r_{12}} & \frac{\partial \sigma_{12}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{12}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{12}}{\partial s_{1}} & \frac{\partial \sigma_{12}}{\partial s_{2}} & \frac{\partial \sigma_{12}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{12}}{\partial s_{k}} \\ \frac{\partial \sigma_{13}}{\partial r_{12}} & \frac{\partial \sigma_{13}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{13}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{13}}{\partial s_{1}} & \frac{\partial \sigma_{13}}{\partial s_{2}} & \frac{\partial \sigma_{13}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{13}}{\partial s_{k}} \\ \vdots & & & & & & & & \vdots \\ \frac{\partial \sigma_{kk}}{\partial r_{12}} & \frac{\partial \sigma_{kk}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{kk}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{kk}}{\partial s_{1}} & \frac{\partial \sigma_{kk}}{\partial s_{2}} & \frac{\partial \sigma_{kk}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{kk}}{\partial s_{k}} \\ \end{vmatrix} \\ &=&\begin{vmatrix} s_1s_2 & 0 & \cdots & 0 & s_2r_{12} & s_1r_{12} & 0 & \cdots & 0\\ 0 & s_1s_3 & \cdots & 0 & s_3r_{13} & 0 & s_1r_{13} & \cdots & 0\\ \vdots & & & & & & & & \vdots \\ 0 & 0 & \cdots & 0 & 0 & 0 & 0 & \cdots & 2s_{k} \end{vmatrix} \end{eqnarray*}

This Jacobian $\mathbf{J}$ can be easily computed by Leibniz formula for determinants

$$ \text{det}(A) = \displaystyle\sum_{\tau \in S_n} \text{sgn}(\tau) \displaystyle\prod_{i=1}^n a_{i,\tau(i)} $$ This formula says that the determinant is the sum taken over all possible permutations of $1,\ldots,n$, that is, the summands are all possible products containing exactly one entry from each row and from each column. The sum has $n!$ terms. The only non-zero summand in that sum (for the Jacobian $\mathbf{J}$) is the product of diagonal elements. For diagonal elements of $\Sigma$ (i.e. $\sigma_{11},\ldots,\sigma_{kk})$ we get $2^k\left(\displaystyle\prod_{i=1}^k s_i\right)$ and for non-diagonal elements of upper triangle of $\Sigma$ (i.e. $\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k}$) we get $\left(\displaystyle\prod_{i=1}^k s_i^{k-1}\right)$, so the Jacobian is $$\mathbf{J} = 2^k \left(\displaystyle\prod_{i=1}^k s_i \right)^k$$ Example: Let's say we have covariance matrix of order $k=2$ $$\Sigma = \begin{pmatrix}\sigma_{11} & \sigma_{12} \\ \sigma_{21} & \sigma_{22}\end{pmatrix}$$ then the Jacobian has the following form $$ \mathbf{J} = \begin{vmatrix} s_1s_2 & s_2r_{12} & s_1r_{12}\\ 0 & 2s_1 & 0\\ 0 & 0 & 2s_2 \end{vmatrix} = 2^2\left(\displaystyle\prod_{i=1}^2s_i\right)^2$$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.