# How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $$\Sigma = \text{diag}(S)\,R\, \text{diag}(S)$$, where $$S$$ is $$k\times 1$$ vector of standard deviations, $$\text{diag}(S)$$ is the diagonal matrix with diagonal elements $$S$$, and $$R$$ is $$k \times k$$ correlation matrix. Elements of the covariance matrix $$\Sigma$$ can be written as follows:

• $$\Sigma_{ii} = s^{2}_{ii}$$
• $$\Sigma_{ij} = s_i s_j r_{ij}$$.

Now let's assume that $$\Sigma$$ has a stadard inverse-Wishart distribution $$W^{-1}_k(I,\nu), \nu\geq k$$,that is, when $$\Sigma$$ has density function $$f_k(\Sigma;I,\nu) \propto |\Sigma|^{-\frac{1}{2}(\nu + k+1)}\, \exp(-\frac{1}{2}\operatorname{tr}(\Sigma^{-1}))$$ Now under the transformation $$\Sigma \rightarrow (S,R)$$ the Jacobian has value of $$2^k\left(\displaystyle \prod _{i}^k s_i\right)^{k}$$. The density function under this transformation is transformed into the following form $$f_k(S,R;I,\nu) \propto |R|^{-\frac{1}{2}(\nu + k+1)} \displaystyle \prod_i^k s_i^{-(\nu+1)} \exp\left(-\frac{r^{ii}}{2 s_i^2}\right)$$ where $$r^{ii}$$ is the $$i$$th diagonal element of $$R^{-1}$$.

Can someone point me in the right direction to compute this Jacobian?

The algebra of differential forms makes short work of this.

I take it that $$\Sigma = (\sigma_{ij})$$ is coordinatized by means of one of its triangles, say the upper triangle with components $$\sigma_{ij},$$ $$1\le i\le j \le k.$$ The differential element of those components, $$|\prod_{ij}\mathrm{d}\sigma_{ij}|,$$ therefore is the (absolute) wedge product of these $$\mathrm{d}\sigma_{ij}.$$ Using

$$\mathrm{d} \sigma_{ii} = 2s_i \mathrm{d}s_i\tag{1}$$

and

$$\mathrm{d} \sigma_{ij} = s_i s_j \mathrm{d} r_{ij} + s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i;\ i \ne j\tag{2}$$

it's surprisingly easy to simplify the differential element in the new coordinates:

\eqalign{ \prod_{ij}\mathrm{d}\sigma_{ij} &= \pm\prod_i \mathrm{d}\sigma_{ii} \wedge \prod_{i\lt j} \mathrm{d}\sigma_{ij} \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i + s_is_j\mathrm{d}r_{ij}\right) \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_is_j\mathrm{d}r_{ij}\right) \\ &= \left(\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j\right)\ \mathrm{d} s_1\wedge \cdots \wedge \mathrm{d}s_k\ \wedge\ \mathrm{d} r_{12} \wedge \cdots \wedge dr_{k-1,k}. }

The second line substituted the differentials $$(1)$$ and $$(2)$$ determined by the transformation. The third line recognized that any term involving $$\mathrm{d}s_i$$ in the right hand product would be multiplied by one of the $$\mathrm{d}s_i$$ in the left hand product, resulting in zero because $$\mathrm{d}s_i\wedge \mathrm{d}s_i=0.$$ The last line merely pulls the constants (the "zero-forms") over to the left.

By definition, the Jacobian is the absolute value of those constants. Evidently each $$s_i$$ appears $$k$$ times. The absolute value is determined from the fact that all the $$s_i$$ are non-negative, yielding

$$|\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j| = 2^k \left(\prod_i s_i\right)^k,$$

QED.

To justify the use of this Jacobian in the Wishart context, we also need to know the transformation between $$\Sigma$$ and $$(S,R)$$ is almost surely one-to-one. But that's clear, because $$(S,R)$$ obviously determines $$\Sigma;$$ while the diagonal of $$\Sigma$$ determines the $$s_i$$ which then can be used to compute the $$r_{ij}$$ from the $$\sigma_{ij}$$ when $$i\ne j.$$

At first I was a little bit puzzled how to transform covariance matrix the way I would transform random variables (e.g. $$Y_1 = X_1X_2$$). Then I realized that covariance matrix $$\Sigma = (\sigma_{ij})$$ of order $$k$$ consists of $$k^2$$ variables and only of $$\frac{k(k+1)}{2}$$ unique variables (i.e. $$\sigma_{ij} = \sigma_{ji}$$), which can be divided into two groups (diagonal elements and let's say upper triangle without the diagonal)

1. $$\sigma_{ii} = s_{ii}^2$$
2. $$\sigma_{ij} = s_is_jr_{ij}$$

So I have $$\frac{k(k+1)}{2}$$-dimensional random variable $$\mathbf{X} = (\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k},\sigma_{11},\ldots,\sigma_{kk})$$ and look for $$\frac{k(k+1)}{2}$$-dimensional random variable $$\mathbf{Y} = (r_{12},\ldots,r_{1k},r_{23},\ldots,r_{2k},\ldots,r_{(k-1)k},s_{11},\ldots,s_{kk})\text{,}$$ where the Jacobian of the transformation has the following form

$$\begin{eqnarray*} \mathbf{J} &=& \begin{vmatrix} \frac{\partial \sigma_{12}}{\partial r_{12}} & \frac{\partial \sigma_{12}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{12}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{12}}{\partial s_{1}} & \frac{\partial \sigma_{12}}{\partial s_{2}} & \frac{\partial \sigma_{12}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{12}}{\partial s_{k}} \\ \frac{\partial \sigma_{13}}{\partial r_{12}} & \frac{\partial \sigma_{13}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{13}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{13}}{\partial s_{1}} & \frac{\partial \sigma_{13}}{\partial s_{2}} & \frac{\partial \sigma_{13}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{13}}{\partial s_{k}} \\ \vdots & & & & & & & & \vdots \\ \frac{\partial \sigma_{kk}}{\partial r_{12}} & \frac{\partial \sigma_{kk}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{kk}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{kk}}{\partial s_{1}} & \frac{\partial \sigma_{kk}}{\partial s_{2}} & \frac{\partial \sigma_{kk}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{kk}}{\partial s_{k}} \\ \end{vmatrix} \\ &=&\begin{vmatrix} s_1s_2 & 0 & \cdots & 0 & s_2r_{12} & s_1r_{12} & 0 & \cdots & 0\\ 0 & s_1s_3 & \cdots & 0 & s_3r_{13} & 0 & s_1r_{13} & \cdots & 0\\ \vdots & & & & & & & & \vdots \\ 0 & 0 & \cdots & 0 & 0 & 0 & 0 & \cdots & 2s_{k} \end{vmatrix} \end{eqnarray*}$$

This Jacobian $$\mathbf{J}$$ can be easily computed by Leibniz formula for determinants

$$\text{det}(A) = \displaystyle\sum_{\tau \in S_n} \text{sgn}(\tau) \displaystyle\prod_{i=1}^n a_{i,\tau(i)}$$ This formula says that the determinant is the sum taken over all possible permutations of $$1,\ldots,n$$, that is, the summands are all possible products containing exactly one entry from each row and from each column. The sum has $$n!$$ terms. The only non-zero summand in that sum (for the Jacobian $$\mathbf{J}$$) is the product of diagonal elements. For diagonal elements of $$\Sigma$$ (i.e. $$\sigma_{11},\ldots,\sigma_{kk})$$ we get $$2^k\left(\displaystyle\prod_{i=1}^k s_i\right)$$ and for non-diagonal elements of upper triangle of $$\Sigma$$ (i.e. $$\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k}$$) we get $$\left(\displaystyle\prod_{i=1}^k s_i^{k-1}\right)$$, so the Jacobian is $$\mathbf{J} = 2^k \left(\displaystyle\prod_{i=1}^k s_i \right)^k$$ Example: Let's say we have covariance matrix of order $$k=2$$ $$\Sigma = \begin{pmatrix}\sigma_{11} & \sigma_{12} \\ \sigma_{21} & \sigma_{22}\end{pmatrix}$$ then the Jacobian has the following form $$\mathbf{J} = \begin{vmatrix} s_1s_2 & s_2r_{12} & s_1r_{12}\\ 0 & 2s_1 & 0\\ 0 & 0 & 2s_2 \end{vmatrix} = 2^2\left(\displaystyle\prod_{i=1}^2s_i\right)^2$$