1
$\begingroup$

I have a Fisher matrix $F$ which has the matrix blocks form like this :

$$ F=\begin{bmatrix} A & B\\ C & D \end{bmatrix} $$

The block $A$ is the most important block, in the sense the parameters of this block are the parameters that I want really get. The block $D$ is called the "nuisance block", that is to say, it consists of the nuisance parameters which deteriorate the informations of the block $A$. The blokc $B$ and $C$ are the correlations blocks between "important block" $A$ and "nuisance block" $D$.

Question : How can I prove mathematically that, by inversing the Fisher matrix $F$, I will get worse constraints (I mean larger variances) on the important parameters (corresponding to $A$ covariance matrix), all of this due to the 3 others blocks ?

Track followed :

I know that (1,1) block for inverse of $F$ : $$ F=\begin{bmatrix} A & B\\ C & D \end{bmatrix} $$

is $(A-BD^{-1}C)^{-1}$ (Schur complement).

From this, if I take the inverse of Fisher matrix $F$, I will get for covariance matrix (the inverse of $F$), the block (1,1) equal to $(A-BD^{-1}C)^{-1}$.

How can we be sure that diagonal of this block $(A-BD^{-1}C)^{-1}$ (which represents the variances of important parameters) will be decreased by the terms "$-BD^{-1}C$" : we should have positive quantities for the diagonal elements of "-$BD^{-1}C$" to make increase the block (1,1) diagonals elements, i.e in order to make worse the constraints, shouldn't we ?

I make the comparison with the values unmarginalised of block A which have the form into initial Fisher matrix F : a_11 = 1/sigma_1^2, a_22 = 1/sigma_2^2, a_33 = 1/sigma_3^2 ... etc. with sigma_i the standard deviation of each important parameters of block A.

I guess it is difficult to prove that nuisance parameters deteriorate the constraints for block (1,1) of covariance matrix but it would be interesting to have a rigorous demonstration.

$\endgroup$
11
  • $\begingroup$ When you say "Fisher matrix", I assume you mean Fisher information matrix. Assuming that is correct, then you must have $B=C^T$ and $A$ and $D$ must be positive definite. With these clarifications, result is very easy to prove. $\endgroup$ Aug 7 '21 at 11:11
  • $\begingroup$ @GordonSmyth . Thanks for your quick answer. Could you explicit please your reasoning in an answer please ? I didn't manage to prove it with your considerations. Regards $\endgroup$
    – youpilat13
    Aug 7 '21 at 11:38
  • $\begingroup$ $C^TD^{-1}C$ is non-negative definite. Result follows immediately. $\endgroup$ Aug 7 '21 at 11:40
  • $\begingroup$ @GordonSmyth . Do you mean that block $(A-BD^{-1}C)$ will be "smaller" than block $A$ ? , so the diagonal elements of $(A-BD^{-1}C)^{-1}$ will be "higher" than $A^{-1}$ (variance will be larger) . It is difficult for me to understand since I am reasoning on diagonal elements whereas I consider all the 4 blocks $A, C^{T}, D$ and $C$, so there are also covariance terms appearing when inversing initial Fisher matrix information $F$. $\endgroup$
    – youpilat13
    Aug 7 '21 at 11:50
  • $\begingroup$ @GordonSmyth . If it doesn't bore you, could you do please a small answer but significant. Regards $\endgroup$
    – youpilat13
    Aug 7 '21 at 17:07
2
+100
$\begingroup$

We know that the inverse of the Fisher information is of the form: $$F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$$ where $(F/A) = D - CA^{-1}B$ is the Schur complement of block $D$.

Let's show that diagonal elements of $A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}$ are bigger than the one of $A^{-1}$, which is equivalent to proving that the diagonal elements of the matrix $R = A^{-1}B(F/A)^{-1}CA^{-1}$ are positive.

Note $p$ the dimension of the upper left block. We'll just proove that for all $e_i$, vector of the canonical basis of $\mathbb{R}^p$, $$e_i^T R e_i \geq 0 .$$ Indeed, as $e_i^T R e_i = R_{ii}$, this will prove what we want.

First note that as $F$ is the Fisher information matrix, it is symmetrical and positive definite. So we get that:

  • $A$ is positive definite,
  • $B^T = C$,
  • Schur complements of $F$, $F/A$ and $F/D$, are symmetrical positive definite.

As $F/A$ is symetrical positive definite, its inverse $(F/A)^{-1}$ also is, and therefore there existe a symetrical definite matrix $Q$ such that $(F/A)^{-1} = Q^T Q$.

Using that, we can write $R$ as $$R = A^{-1} C^T Q^T QCA^{-1} = \left(Q C A^{-1}\right)^T\left(QCA^{-1}\right)$$

Therefore $$e_i^T R e_i = (QCA^{-1}e_i)^T(QCA^{-1}e_i) = \lVert QCA^{-1}e_i\rVert_2 \geq 0. $$

Hence the result.

Hope this is useful.


Additional note : Proof that $F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$

Write $F$ as $$F = \left[ \begin{array}{cc} A & B \\ C & D \end{array}\right] = \left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right] \left[ \begin{array}{cc} A & 0 \\ 0 & D - C A^{-1} B \end{array}\right] \left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]$$ Such that $$ F^{-1} = \left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]^{-1} \left[ \begin{array}{cc} A & 0 \\ 0 & D - C A^{-1} B \end{array}\right]^{-1} \left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right]^{-1}. $$ It's easy to check that $$\left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right]^{-1}=\left[ \begin{array}{cc} I_p & 0 \\ -CA^{-1} & I_q \end{array}\right]$$ and

$$\left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]^{-1}=\left[ \begin{array}{cc} I_p & -A^{-1}B \\ 0 & I_q \end{array}\right]$$ and

$$\left[ \begin{array}{cc} A & 0 \\ 0 & D - CA^{-1}B \end{array}\right]^{-1}=\left[ \begin{array}{cc} A^{-1} & 0 \\ 0 & (D - CA^{-1}B)^{-1} \end{array}\right].$$

Therefore $$ F^{-1} = \left[ \begin{array}{cc} I_p & -A^{-1}B \\ 0 & I_q \end{array}\right] \left[ \begin{array}{cc} A^{-1} & 0 \\ 0 & (D - C A^{-1} B)^{-1} \end{array}\right] \left[ \begin{array}{cc} I_p & 0 \\ -CA^{-1} & I_q \end{array}\right] = \left[ \begin{array}{cc} A^{−1} + A^{−1}B (F/A)^{−1}CA^{-1} & −A^{−1}B(F/A)^{−1}\\ −(F/A)^{−1}CA^{−1} & (F/A)^{−1} \end{array}\right] $$ where $F/A = D - CA^{-1}B$ .

$\endgroup$
9
  • $\begingroup$ Thanks, I am going to get though it. Just a detail, a closed $are missing in (F/A)^{-1}. $\endgroup$
    – youpilat13
    Oct 7 '21 at 13:09
  • $\begingroup$ Are you sure that your expression is correct when you write : $$F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$$ Isn't it rather for the first block : $$(A-BD^{-1}C)^{-1}$$ ? $\endgroup$
    – youpilat13
    Oct 7 '21 at 15:32
  • $\begingroup$ I found this on the Wikipedia page of Schur's complement en.wikipedia.org/wiki/Schur_complement#Properties . I guess then $(A - BD^{-1}C)^{-1} = A^{-1} + A^{-1} B (F/A)^{-1}C A^{-1}$... $\endgroup$
    – Pohoua
    Oct 7 '21 at 16:48
  • $\begingroup$ Thanks. I understand the first expression below but not the development which allows to get the second expression of $M^{-1}$ : $\endgroup$
    – youpilat13
    Oct 7 '21 at 19:18
  • $\begingroup$ $$ $$- In general, if $A$ is invertible, then $$ \begin{aligned} M &=\left[\begin{array}{ll} A & B \\ C & D \end{array}\right]=\left[\begin{array}{cc} I_{p} & 0 \\ C A^{-1} & I_{q} \end{array}\right]\left[\begin{array}{cc} A & 0 \\ 0 & D-C A^{-1} B \end{array}\right]\left[\begin{array}{cc} I_{p} & A^{-1} B \\ 0 & I_{q} \end{array}\right] \\ M^{-1} &=\left[\begin{array}{cc} A^{-1}+A^{-1} B(M / A)^{-1} C A^{-1} & -A^{-1} B(M / A)^{-1} \\ & -(M / A)^{-1} C A^{-1} & (M / A)^{-1} \end{array}\right] \end{aligned} $$ Could you help please to do the development to get $M^{-1}$ ? $\endgroup$
    – youpilat13
    Oct 7 '21 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.