How can nuisance parameters in Fisher matrix can deteriorate the useful constraints?

I have a Fisher matrix $$F$$ which has the matrix blocks form like this :

$$F=\begin{bmatrix} A & B\\ C & D \end{bmatrix}$$

The block $$A$$ is the most important block, in the sense the parameters of this block are the parameters that I want really get. The block $$D$$ is called the "nuisance block", that is to say, it consists of the nuisance parameters which deteriorate the informations of the block $$A$$. The blokc $$B$$ and $$C$$ are the correlations blocks between "important block" $$A$$ and "nuisance block" $$D$$.

Question : How can I prove mathematically that, by inversing the Fisher matrix $$F$$, I will get worse constraints (I mean larger variances) on the important parameters (corresponding to $$A$$ covariance matrix), all of this due to the 3 others blocks ?

Track followed :

I know that (1,1) block for inverse of $$F$$ : $$F=\begin{bmatrix} A & B\\ C & D \end{bmatrix}$$

is $$(A-BD^{-1}C)^{-1}$$ (Schur complement).

From this, if I take the inverse of Fisher matrix $$F$$, I will get for covariance matrix (the inverse of $$F$$), the block (1,1) equal to $$(A-BD^{-1}C)^{-1}$$.

How can we be sure that diagonal of this block $$(A-BD^{-1}C)^{-1}$$ (which represents the variances of important parameters) will be decreased by the terms "$$-BD^{-1}C$$" : we should have positive quantities for the diagonal elements of "-$$BD^{-1}C$$" to make increase the block (1,1) diagonals elements, i.e in order to make worse the constraints, shouldn't we ?

I make the comparison with the values unmarginalised of block A which have the form into initial Fisher matrix F : a_11 = 1/sigma_1^2, a_22 = 1/sigma_2^2, a_33 = 1/sigma_3^2 ... etc. with sigma_i the standard deviation of each important parameters of block A.

I guess it is difficult to prove that nuisance parameters deteriorate the constraints for block (1,1) of covariance matrix but it would be interesting to have a rigorous demonstration.

• When you say "Fisher matrix", I assume you mean Fisher information matrix. Assuming that is correct, then you must have $B=C^T$ and $A$ and $D$ must be positive definite. With these clarifications, result is very easy to prove. Aug 7 '21 at 11:11
• $C^TD^{-1}C$ is non-negative definite. Result follows immediately. Aug 7 '21 at 11:40
• @GordonSmyth . Do you mean that block $(A-BD^{-1}C)$ will be "smaller" than block $A$ ? , so the diagonal elements of $(A-BD^{-1}C)^{-1}$ will be "higher" than $A^{-1}$ (variance will be larger) . It is difficult for me to understand since I am reasoning on diagonal elements whereas I consider all the 4 blocks $A, C^{T}, D$ and $C$, so there are also covariance terms appearing when inversing initial Fisher matrix information $F$. Aug 7 '21 at 11:50
• @GordonSmyth . If it doesn't bore you, could you do please a small answer but significant. Regards Aug 7 '21 at 17:07

We know that the inverse of the Fisher information is of the form: $$F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$$ where $$(F/A) = D - CA^{-1}B$$ is the Schur complement of block $$D$$.

Let's show that diagonal elements of $$A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}$$ are bigger than the one of $$A^{-1}$$, which is equivalent to proving that the diagonal elements of the matrix $$R = A^{-1}B(F/A)^{-1}CA^{-1}$$ are positive.

Note $$p$$ the dimension of the upper left block. We'll just proove that for all $$e_i$$, vector of the canonical basis of $$\mathbb{R}^p$$, $$e_i^T R e_i \geq 0 .$$ Indeed, as $$e_i^T R e_i = R_{ii}$$, this will prove what we want.

First note that as $$F$$ is the Fisher information matrix, it is symmetrical and positive definite. So we get that:

• $$A$$ is positive definite,
• $$B^T = C$$,
• Schur complements of $$F$$, $$F/A$$ and $$F/D$$, are symmetrical positive definite.

As $$F/A$$ is symetrical positive definite, its inverse $$(F/A)^{-1}$$ also is, and therefore there existe a symetrical definite matrix $$Q$$ such that $$(F/A)^{-1} = Q^T Q$$.

Using that, we can write $$R$$ as $$R = A^{-1} C^T Q^T QCA^{-1} = \left(Q C A^{-1}\right)^T\left(QCA^{-1}\right)$$

Therefore $$e_i^T R e_i = (QCA^{-1}e_i)^T(QCA^{-1}e_i) = \lVert QCA^{-1}e_i\rVert_2 \geq 0.$$

Hence the result.

Hope this is useful.

Additional note : Proof that $$F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$$

Write $$F$$ as $$F = \left[ \begin{array}{cc} A & B \\ C & D \end{array}\right] = \left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right] \left[ \begin{array}{cc} A & 0 \\ 0 & D - C A^{-1} B \end{array}\right] \left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]$$ Such that $$F^{-1} = \left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]^{-1} \left[ \begin{array}{cc} A & 0 \\ 0 & D - C A^{-1} B \end{array}\right]^{-1} \left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right]^{-1}.$$ It's easy to check that $$\left[ \begin{array}{cc} I_p & 0 \\ CA^{-1} & I_q \end{array}\right]^{-1}=\left[ \begin{array}{cc} I_p & 0 \\ -CA^{-1} & I_q \end{array}\right]$$ and

$$\left[ \begin{array}{cc} I_p & A^{-1}B \\ 0 & I_q \end{array}\right]^{-1}=\left[ \begin{array}{cc} I_p & -A^{-1}B \\ 0 & I_q \end{array}\right]$$ and

$$\left[ \begin{array}{cc} A & 0 \\ 0 & D - CA^{-1}B \end{array}\right]^{-1}=\left[ \begin{array}{cc} A^{-1} & 0 \\ 0 & (D - CA^{-1}B)^{-1} \end{array}\right].$$

Therefore $$F^{-1} = \left[ \begin{array}{cc} I_p & -A^{-1}B \\ 0 & I_q \end{array}\right] \left[ \begin{array}{cc} A^{-1} & 0 \\ 0 & (D - C A^{-1} B)^{-1} \end{array}\right] \left[ \begin{array}{cc} I_p & 0 \\ -CA^{-1} & I_q \end{array}\right] = \left[ \begin{array}{cc} A^{−1} + A^{−1}B (F/A)^{−1}CA^{-1} & −A^{−1}B(F/A)^{−1}\\ −(F/A)^{−1}CA^{−1} & (F/A)^{−1} \end{array}\right]$$ where $$F/A = D - CA^{-1}B$$ .

• Thanks, I am going to get though it. Just a detail, a closed $are missing in (F/A)^{-1}. Oct 7 '21 at 13:09 • Are you sure that your expression is correct when you write : $$F^{-1} = \left[ \begin{array}{cc} A^{-1}+A^{-1}B(F/A)^{-1}CA^{-1}& ... \\ ... & ... \end{array}\right]$$ Isn't it rather for the first block : $$(A-BD^{-1}C)^{-1}$$ ? Oct 7 '21 at 15:32 • I found this on the Wikipedia page of Schur's complement en.wikipedia.org/wiki/Schur_complement#Properties . I guess then$(A - BD^{-1}C)^{-1} = A^{-1} + A^{-1} B (F/A)^{-1}C A^{-1}$... Oct 7 '21 at 16:48 • Thanks. I understand the first expression below but not the development which allows to get the second expression of$M^{-1}$: Oct 7 '21 at 19:18 • - In general, if$Ais invertible, then \begin{aligned} M &=\left[\begin{array}{ll} A & B \\ C & D \end{array}\right]=\left[\begin{array}{cc} I_{p} & 0 \\ C A^{-1} & I_{q} \end{array}\right]\left[\begin{array}{cc} A & 0 \\ 0 & D-C A^{-1} B \end{array}\right]\left[\begin{array}{cc} I_{p} & A^{-1} B \\ 0 & I_{q} \end{array}\right] \\ M^{-1} &=\left[\begin{array}{cc} A^{-1}+A^{-1} B(M / A)^{-1} C A^{-1} & -A^{-1} B(M / A)^{-1} \\ & -(M / A)^{-1} C A^{-1} & (M / A)^{-1} \end{array}\right] \end{aligned} Could you help please to do the development to getM^{-1}\$ ? Oct 7 '21 at 19:19