The probability limit of an inverse matrix

all:

I am considering a question regarding the calculation of the probability limit for an inverse matrix.

Specifically, suppose we have a non-singular and squared matrix $M$ with dimension $2\times 2$, and let $a_{ij}$ be the $ij$th element in $M$.

Assume that for all i and j = 1,2, the probability limit of $a_{ij}$ is $a_{ij}=a_{0,ij}+o(1)$, where the term $o(1)$ indicates that the remaining term goes to zero asymptotically. In this case we can state that $M=M_0+o(1)$, where the $ij$th element in $M_0$ is $a_{0,ij}$.

Now my question is the following. Let $M^{-1}$ be the inverse matrix of $M$. Given that $M=M_0+o(1)$, can we claim that $M^{-1}=M_0^{-1}+o(1)$? If this is true, then the calculation of $M^{-1}$ can be greatly simplified, especially at higher dimension.

I am not sure if this is a well-established fact or it is simply my claim. I would be appreciate if you can share with me your thought on this issue, or let me know any related paper/theory/lemma that you might know.

Thank you in advance. If the question is not clear please also let me know.

• As evidence concerning how well-established this might be, note that applying the Binomial Theorem to $M_0+o(1)$ (for the $-1$ power) in the form \eqalign{(M_0+o(1))^{-1}&=(M_0(1+M_0^{-1}o(1)))^{-1}\\&=(1+M_0^{-1}o(1))^{-1}M_0^{-1}\\&=(1-M_0^{-1}o(1))M_0^{-1} \\&=M_0^{-1}-M_0^{-2}o(1)}immediately gives you $M^{-1}=M_0^{-1}+o(1)$, as well as showing how large the potential error might be. Carrying the Binomial expansion beyond $o(1)$ will give you additional correction terms. – whuber Aug 16 '17 at 19:52
• sorry if I miss something, but how the equation 2 and 3 are equal? or, why (1+inverse(M0)o(1))^(-1)=(1-inverse(M0)o(1))^(-1)? – Rico Aug 16 '17 at 20:31
• That's precisely what the Binomial Theorem implies. It's easy to verify: since $$(1-M_0^{-1}o(1))(1+M_0^{-1}o(1))=1+o(1)^2\cong 1\ \operatorname{mod}(o(1)),$$ the expression I gave for the inverse really works. – whuber Aug 16 '17 at 21:56
• @whuber after searching related materials I gauss I need to learn from you for the following two questions. If we have two matrix $A$ and $B$ that are commute, then I think we can apply the standard binominal theorem here. Now we do have $M_0$ and o(1) that are commute, but Q1: how would we apply binomial theorem with power $-1$ to our case here? and Q2: what do you mean by $mod(o(1))$ here? sorry if I missed too much. – Rico Aug 16 '17 at 22:52
• You are right to pay attention to commuting--but here we are interested only in evaluating the sizes of matrices. For that purpose, "$o(1)$" commutes with everything. If your purpose is to compute correction terms, then you have to use the approximation $$(M+\epsilon)^{-1}\approx M^{-1}-M^{-1}\epsilon M^{-1}$$ which is valid for sufficiently small matrices $\epsilon$. Finally, $\operatorname{mod}(o(1))$ means that all terms of size $o(1)$ or smaller are ignored. – whuber Aug 16 '17 at 23:01

I'm going to change your notation a little bit to avoid so many subscripts and to make things clearer. It seems that you're considering a sequence of random matrices $$M_n = \begin{bmatrix} a_n & b_n \\ c_n & d_n\end{bmatrix}$$ where

$$M_n \to_p M := \begin{bmatrix} a & b \\ c & d\end{bmatrix}$$ and convergence is element-wise.

Your question seems to boil down to $M_n^{-1} \stackrel ?{\to_p} M^{-1}$.

Because $M_n$ is $2\times 2$ we know that $$M_n^{-1} = \frac{1}{\det M_n} \begin{bmatrix} d_n & -b_n \\ -c_n & a_n\end{bmatrix}$$ and $$M^{-1} = \frac{1}{\det M} \begin{bmatrix} d & -b \\ -c & a\end{bmatrix}.$$

First off, consider $\det M_n = a_nd_n - b_n c_n$. Since all four of our random variable sequences converge to constants, we can use Slutsky's theorem to show that $\det M_n \to_p ad-bc = \det M$.

We also know (again by Slutsky) that $a_n / g_n \to_p a / g$ for some random sequence $g_n \to g$ provided $g_n, g \neq 0$ so $a_n / \det M_n \to a / \det M$ since we're assuming all $M_n$ are invertible along with $M$, so the determinants are never $0$. An analogous argument shows that this holds for all elements of $M_n^{-1}$.

In summary: Basically I'm using successive applications of Slutky's theorem to show that if $X_{ij} \to_p \mu_i$ then $\frac{\pm X_{ij}}{X_{1j}X_{4j} - X_{2j}X_{3j}} \to_p \frac{\pm \mu_i}{\mu_1\mu_4 - \mu_2\mu_3}$ provided the denominators are never $0$.

Another way to look at this is that the mapping $M_n \mapsto M_n^{-1}$ is continuous, provided the inverse exists, so by the continuous mapping theorem limits are preserved.

• Thank you very much for the quick response, Chaconne. Your notation is very clear to me, and yes you understand my question completely. In fact, when I write this post I wish to start from a simple example where the random matrix Mn is has dimension 2 by 2. In general, I am considering a random block matrix Mn with blocks An, Bn, Cn, and Dn. Each block converges in probability to A+o(1), B+o(1), C+o(1), and D+o(1). In other words, Mn=M+o(1), where M is the block matrix with component A, B, C, and D. – Rico Aug 16 '17 at 19:51
• The difference now is that we need to compute the inverse of Mn using the block matrix inverse formula. Do you think we can still apply Slutsky's theorm on answer my question? – Rico Aug 16 '17 at 19:54
• One more question I just missed: when using slutsky's theorm, can we claim that the inverse of Mn converges in probability to the inverse of M+o(1)? (i.e., can the term o(1) be remained?) – Rico Aug 16 '17 at 20:39
• @Rico $M_n^{-1}$ converges exactly in probability to $M^{-1}$, there's no $o(1)$ in the limit. As for the block matrix case, I think these are all continuous functions so the continuous mapping theorem should still give us the result, but it'd be worth checking more carefully – jld Aug 16 '17 at 22:37
• Thanks for the help and I will check. I do have a concern regarding applying Slushy theorem. It states that for two random variables $X_n$ and $Y_n$, if $X_n$ converges to $X$ in distribution and $Y_n$ converges to a constant $c$ in probability, then the result are all for convergence in distribution, such as $det M_n$ converges to $det M$ in distribution, not in probability. However, I need to demonstrate convergence in probability. We know that convergence in distribution does not necessarily imply convergence in probability. Is there something I missed? – Rico Aug 16 '17 at 22:44