Consider the simple linear regression model. $$ y_i = \beta_0 + \beta_1 x_i + \varepsilon_i, \quad \quad \quad \quad i = 1,2,\dots,n. $$ Let $\mu_x$ and $\sigma_x^2$ represent the mean and variance of the i.i.d. observations $x_1,x_2,\dots,x_n$. The simple linear regression model can be written in matrix form as $$ y = \mathbf{X} \beta + \varepsilon, $$ where $$ \mathbf{X} = \begin{bmatrix} \mathbf{x_1}^T \\ \mathbf{x_2}^T \\ \vdots \\ \mathbf{x_n}^T \end{bmatrix} = \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \\ 1 & x_n \end{bmatrix}. $$
Regarding the large sample theory for regression, it is known that $$ \bigg(\frac{\mathbf{X}^T \mathbf{X}}{n}\bigg) \stackrel{p}{\longrightarrow} E[\mathbf{x}_i \mathbf{x}_i']. $$ This is shown in equation (6.1) in these notes, where the author uses the notation \begin{align} \hat Q_{xx} = \bigg(\frac{\mathbf{X}^T \mathbf{X}}{n}\bigg), \\ Q_{xx} = E[\mathbf{x}_i \mathbf{x}_i']. \end{align}
Sticking with the notation in those notes, we then have $$ \hat Q_{xx}^{-1} \stackrel{p}{\longrightarrow} Q_{xx}^{-1}. $$ I want to find the actual rate of convergence in probability of $\hat Q_{xx}^{-1}$ to $Q_{xx}^{-1}$. The above expression means that $\hat Q_{xx}^{-1} - Q_{xx}^{-1} = o_p(1)$, but I am hoping that the convergence can be shown more precisely or by shown to be faster, possibly something like $\hat Q_{xx}^{-1} - Q_{xx}^{-1} = O_p(n^{-1/2})$ or $o_p(n^{-1/2})$.
Since we are considering simple linear regression we have an explicit representation for $\hat Q_{xx}^{-1}$: $$ \begin{align} \hat Q_{xx}^{-1} = n (\mathbf{X}^T \mathbf{X})^{-1} & = \frac{1}{\sum_{i=1}^n(x_i - \overline x)^2} \begin{bmatrix} \sum_{i=1}^n x_i^2 & - \sum_{i=1}^n x_i \\ - \sum_{i=1}^n x_i & n \end{bmatrix}, \\ & = \frac{1}{\frac{1}{n}\sum_{i=1}^n(x_i - \overline x)^2} \begin{bmatrix} \frac{1}{n} \sum_{i=1}^n x_i^2 & - \frac{1}{n} \sum_{i=1}^n x_i \\ - \frac{1}{n} \sum_{i=1}^n x_i & \frac{1}{n} n \end{bmatrix}, \\ & = \frac{1}{s^2} \begin{bmatrix} \gamma & - \overline x \\ - \overline x & 1 \end{bmatrix}. \end{align} $$
Then we have:
- $\sqrt{n}(s^2 - \sigma_x^2) \stackrel{d}{\longrightarrow} \mathcal{N}(0,E[x_i - \mu_x^4])$ (shown here)
- $\sqrt{n}(\overline x - \mu_x) \stackrel{d}{\longrightarrow} \mathcal{N}(0,\sigma_x^2)$
- $\sqrt{n}(\gamma - (?)) \stackrel{d}{\longrightarrow} \mathcal{N}(0,(?))$ (I'm not sure about this one)
In any, case the numerator and denominator of each element of the $2\times 2$ matrix $\hat Q_{xx}^{-1}$ is converging in distribution, and since they are converging to a constant, we get convergence in probability. Due to the $\sqrt{n}$ term at the front of each expression it appears we can say that the convergence of $\hat Q_{xx}^{-1}$ to $Q_{xx}^{-1}$ really is faster than $o_p(1)$. It appears we can say $$ \hat Q_{xx}^{-1} - Q_{xx}^{-1} = O_p(n^{-1/2}). $$
Is this in fact the case, do we really have a rate of convergence of $O_p(n^{-1/2})$ or have I made a mistake?