3 replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
source | link

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. (A simple example of its utility in statistics appears at http://stats.stackexchange.com/questions/120459/diagonal-elements-of-the-inverted-correlation-matrix/120476#120476Diagonal elements of the inverted correlation matrix.)

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. (A simple example of its utility in statistics appears at http://stats.stackexchange.com/questions/120459/diagonal-elements-of-the-inverted-correlation-matrix/120476#120476.)

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. (A simple example of its utility in statistics appears at Diagonal elements of the inverted correlation matrix.)

2 added 339 characters in body
source | link

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. Although these operations are (A simple example of its utility in statistics appears at http://stats.stackexchange.com/questions/120459/diagonal-elements-of-the-inverted-correlation-matrix/120476#120476.)

Although this operation is well-known and described elsewhere on the Web, it is not easy to find descriptions that are sufficiently clear and general that they indicate how to generalize themthey might apply to reduction of block matrices. Let's fill that gap and then apply the result to get some insight into the present problem.

The idea is to apply block row operations until the determinant of the matrix can easily be calculated. Applying this to Considering the matrix $X$ in the question, we can immediately see how to eliminate the elements along the bottom row by removing multiples of the first, second, and third rows (in any order). This will change the matrix to

This isThese operations are feasible because

The determinant of the resulting block triangular matrix is the product of the determinants of the blocks along the diagonal:The determinant of the resulting block triangular matrix is the product of the determinants of the blocks along the diagonal:

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. Although these operations are well-known and described elsewhere on the Web, it is not easy to find descriptions that are sufficiently clear and general that they indicate how to generalize them to reduction of block matrices.

The idea is to apply block row operations until the determinant of the matrix can easily be calculated. Applying this to the matrix $X$ in the question, we can immediately see how to eliminate the elements along the bottom row. This will change the matrix to

This is feasible because

The determinant of the resulting block triangular matrix is the product of the determinants of the blocks along the diagonal:

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. (A simple example of its utility in statistics appears at http://stats.stackexchange.com/questions/120459/diagonal-elements-of-the-inverted-correlation-matrix/120476#120476.)

Although this operation is well-known and described elsewhere on the Web, it is not easy to find descriptions that are sufficiently clear and general that they indicate how they might apply to block matrices. Let's fill that gap and then apply the result to get some insight into the present problem.

The idea is to apply block row operations until the determinant of the matrix can easily be calculated. Considering the matrix $X$ in the question, we can immediately see how to eliminate the elements along the bottom row by removing multiples of the first, second, and third rows (in any order). This will change the matrix to

These operations are feasible because

The determinant of the resulting block triangular matrix is the product of the determinants of the blocks along the diagonal:

1
source | link

Let's talk a little about a useful way to manipulate block matrices: row reduction or Gaussian elimination. Although these operations are well-known and described elsewhere on the Web, it is not easy to find descriptions that are sufficiently clear and general that they indicate how to generalize them to reduction of block matrices.

Let, then, $m = m_1 + m_2 + m_3 + m_4$ be the number of rows in any matrix $X$ and $n = n_1 + n_2 + n_3 + n_4$ be the number of its columns. We may write

$$X = \pmatrix{X_{11} & X_{12} & X_{13} & X_{14} \\ X_{21} & X_{22} & X_{23} & X_{24} \\ X_{31} & X_{32} & X_{33} & X_{34} \\ X_{41} & X_{42} & X_{43} & X_{44}}$$

where the $X_{ij}$ are $m_i\times n_j$ matrices (and any of the $m_i$ or $n_j$ may be zero).

Suppose you can find a matrix $U$ for which $U X_{11} = X_{31}$ (so that $U$ must be an $m_3 \times m_1$ matrix). You may use it to "eliminate" $X_{31}$ via a "block row operation":

$$\pmatrix{1_{m_1} & 0 & 0 & 0 \\ 0 & 1_{m_2} & 0 & 0 \\ -U & 0 & 1_{m_3} & 0 \\ 0 & 0 & 0 & 1_{m_4}} X = \pmatrix{X_{11} & X_{12} & X_{13} & X_{14} \\ X_{21} & X_{22} & X_{23} & X_{24} \\ 0 & X_{32} - U X_{12} & X_{33} - U X_{13} & X_{34} - U X_{14} \\ X_{41} & X_{42} & X_{43} & X_{44}}.$$

The notation "$1_k$" refers to the $k\times k$ identity matrix.

It is obvious that the determinant of the "elimination matrix" on the left hand side is $1$. Therefore, whenever $X$ is a square matrix, a row block operation does not change the determinant of the result.

The idea is to apply block row operations until the determinant of the matrix can easily be calculated. Applying this to the matrix $X$ in the question, we can immediately see how to eliminate the elements along the bottom row. This will change the matrix to

$$\pmatrix{A & 0 & 0 & E \\ 0 & B & 0 & F \\ 0 & 0 & C & G \\ 0 & 0 & 0 & D - E^\prime A^{-1} E - F^\prime B^{-1} F - G^\prime C^{-1} G}.$$

This is feasible because

  • The positive-definiteness of $A$, $B$, and $C$ implies all three are invertible.

  • The unique solution to $E^\prime = UA$ is $E^\prime A^{-1}$, the unique solution to $F^\prime = UB$ is $F^\prime B^{-1}$, and the unique solution to $G^\prime = UC$ is $G^\prime C^{-1}$.

The determinant of the resulting block triangular matrix is the product of the determinants of the blocks along the diagonal:

$$|X| = |A|\,|B|\,|C|\,|D - E^\prime A^{-1} E - F^\prime B^{-1} F - G^\prime C^{-1} G|.$$

As a check, let X be the matrix defined in the R code in the question. Let's compare the result of this formula to the direct calculation of the determinant:

A <- X[1:2, 1:2]
B <- X[3:4, 3:4]
C <- X[5:6, 5:6]
D <- X[7:8, 7:8]
E <- X[1:2, 7:8]
F <- X[3:4, 7:8]
G <- X[5:6, 7:8]
det.X <- det(X)
det.X0 <- det(A) * det(B) * det(C) * det(D - crossprod(E, solve(A, E))
                                           - crossprod(F, solve(B, F))
                                           - crossprod(G, solve(C, G)))

(det.X - det.X0) / det.X

[1] -1.926411e-16

The relative error is the size of floating point roundoff error: the two values have to be considered equal.

For small blocks, this procedure is primarily of theoretical interest. In R, it takes 50 times longer to carry out the calculation based on blocks than it does to compute the determinant directly!