# Is $\left\{(\mathbf{X_n}^T\mathbf{X_n}/n)^{-1}\right\}_{n=1}^\infty$ uniformly integrable (UI)? What assumptions make it UI?

$$\left\{(\mathbf{X_n}^T\mathbf{X_n}/n)^{-1}\right\}_{n=1}^\infty$$

Let $$\mathbf{X}_n$$ be the usual data matrix in standard multiple regression where I have used the subscript $$n$$ to indicate the number of observations: $$\mathbf{X}_n = \begin{bmatrix} \mathbf{x}_1^T \\ \mathbf{x}_2^T \\ \vdots \\ \mathbf{x}_n^T \\ \end{bmatrix} = \begin{bmatrix} 1 & x_{11} & \dots & x_{1P} \\ 1 & x_{21} & \dots & x_{2P} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{n1} & \dots & x_{nP} \\ \end{bmatrix}$$

Under the standard assumptions of multiple linear regression, the conditional variance of the estimated least squares regression coefficients $$\hat \beta$$ is $$\text{Var}[\hat \beta| \mathbf{X_n}] = \sigma^2(\mathbf{X_n}^T \mathbf{X_n})^{-1},$$ where $$\sigma^2$$ is the variance of the error.

One of the assumptions of linear regression for large samples is $$\text{plim}_{n\to \infty }\bigg(\frac{\mathbf{X_n}^T \mathbf{X_n}}{n}\bigg)^{-1} = Q,$$ a positive definite matrix.

Define the sequence of random matrices $$\{Y_n\}_{n=1}^\infty$$ where $$Y_n := \bigg(\frac{\mathbf{X_n}^T \mathbf{X_n}}{n}\bigg)^{-1}.$$

I am interested in the uniform integrability (UI) of the elements of $$Y_n$$. Namely because if they are uniformly integrable, then we can say the elements of $$Y_n$$ converge in expectation to the elements of $$Q$$, since convergence in probability + UI implies convergence in expectation.

Is it possible that $$\{Y_n\}_{n=1}^\infty$$ is automatically uniformly integrable due to the fact that the observations $$\mathbf{x}_i$$ used to create the data matrix are i.i.d.?

If not, what conditions should we impose on the (elements of the) data matrix $$\mathbf{X_n}$$ so that we obtain uniform integrability for the (elements of the) sequence of matrices $$\{Y_n\}_{n=1}^\infty$$

I am not sure if it is possible to give a thorough answer here (at least for me) given the breadth of the possible issues (dependence, heterogeneity, existence of moments etc.).

That said, here are some thoughts:

Consider the case $$P=1$$ and no constant for simplicity. Then, letting $$Y_n:=(Z_n)^{-1}$$, where $$Z_n=\frac{\sum_{i=1}^nx_i^2}{n}.$$ We have $$E(Z_n)=E\left(\frac{\sum_{i=1}^nx_i^2}{n}\right)=\frac{\sum_{i=1}^nE(x_i^2)}{n}.$$ If the $$x_i$$ are iid, this simplifies to $$E(x_i^2)$$.

Uniform integrability is usually invoked to shut down counterexamples like this one: if $$x_i=i$$ w.p. $$1/i$$ and zero w.p. $$1-1/i$$, we have $$x_i\to_p0$$ but $$E(x_i)=1$$ for any $$i$$, so expectations do not converge (to the plim). Also, $$E(x_i^2)=i$$, which diverges. Then, $$E(Z_n)=\frac{n(n+1)}{2n}\to\infty$$ This, however, is evidently not an i.i.d. setup.

Davidson, Stochastic Limit Theory, Theorem 12.11, for example provides sufficient conditions for uniform integrability, such as uniformly bounded moments. That is $$\{X_t\}$$ is uniformly integrable if $$E|X_t|^{1+\theta}<\infty$$ for $$\theta>0$$.

• Interesting post. Yes I noticed that the examples for when convergence in probability doesn't match convergence in expectation seem to always make a random variable $X_n$ under consideration heavily dependent on the index $n$ of the sequence (such as the image on the Wikipedia page. But for 'conventional' multiple regression in the real world an observation isn't dependent on the particular index $n$ of the sequence of observations. – sonicboom Dec 10 '20 at 17:24
• For example if we regress heights against weights for all people in a particular city on a particular day. Then the rows of the data matrix $\mathbf{X}_n$ will feature observations of individual people's weights, and these will not be dependent on $n$. They are simply i.i.d observations. Its for this type of reason that I wonder if the $\{Y_n\}_{n=1}^\infty$ in my original question could be automatically uniformly integrable due to the i.i.d. assumption. – sonicboom Dec 10 '20 at 17:28
• Indeed, as I was trying to express (not very well, apparently), I could not come up with an issue when there is iid-ness, and looking at the definition of uniform integrability suggests that it is relevant basically when there indeed is heterogeneity. – Christoph Hanck Dec 10 '20 at 18:17
• Section 2 of this paper seems to imply the OLS estimators are uniformly integrable which I assume would mean $\left\{(\mathbf{X_n}^T\mathbf{X_n}/n)^{-1}\right\}_{n=1}^\infty$ is UI since $(\mathbf{X_n}^T\mathbf{X_n})^{-1}$ is used in the construction of the OLS estimators. – sonicboom Dec 13 '20 at 16:29
• Actually I think that paper uses fixed regressors not stochastic regressors so it does not correspond to the case I am asking about. – sonicboom Mar 15 at 10:04