Let $\mathbf{X}_n$ be the usual data matrix in standard multiple regression where I have used the subscript $n$ to indicate the number of observations: $$ \mathbf{X}_n = \begin{bmatrix} \mathbf{x}_1^T \\ \mathbf{x}_2^T \\ \vdots \\ \mathbf{x}_n^T \\ \end{bmatrix} = \begin{bmatrix} 1 & x_{11} & \dots & x_{1P} \\ 1 & x_{21} & \dots & x_{2P} \\ \vdots & \vdots & \ddots & \vdots \\ 1 & x_{n1} & \dots & x_{nP} \\ \end{bmatrix} $$

Under the standard assumptions of multiple linear regression, the conditional variance of the estimated least squares regression coefficients $\hat \beta$ is $$ \text{Var}[\hat \beta| \mathbf{X_n}] = \sigma^2(\mathbf{X_n}^T \mathbf{X_n})^{-1}, $$ where $\sigma^2$ is the variance of the error.

One of the assumptions of linear regression for large samples is $$ \text{plim}_{n\to \infty }\bigg(\frac{\mathbf{X_n}^T \mathbf{X_n}}{n}\bigg)^{-1} = Q, $$ a positive definite matrix.

Define the sequence of random matrices $\{Y_n\}_{n=1}^\infty$ where $$ Y_n := \bigg(\frac{\mathbf{X_n}^T \mathbf{X_n}}{n}\bigg)^{-1}. $$

I am interested in the uniform integrability (UI) of the elements of $Y_n$. Namely because if they are uniformly integrable, then we can say the elements of $Y_n$ converge in expectation to the elements of $Q$, since convergence in probability + UI implies convergence in expectation.

Is it possible that $\{Y_n\}_{n=1}^\infty$ is automatically uniformly integrable due to the fact that the observations $\mathbf{x}_i$ used to create the data matrix are i.i.d.?

If not, what conditions should we impose on the (elements of the) data matrix $\mathbf{X_n}$ so that we obtain uniform integrability for the (elements of the) sequence of matrices $\{Y_n\}_{n=1}^\infty$


I am not sure if it is possible to give a thorough answer here (at least for me) given the breadth of the possible issues (dependence, heterogeneity, existence of moments etc.).

That said, here are some thoughts:

Consider the case $P=1$ and no constant for simplicity. Then, letting $Y_n:=(Z_n)^{-1}$, where $$ Z_n=\frac{\sum_{i=1}^nx_i^2}{n}. $$ We have $$ E(Z_n)=E\left(\frac{\sum_{i=1}^nx_i^2}{n}\right)=\frac{\sum_{i=1}^nE(x_i^2)}{n}. $$ If the $x_i$ are iid, this simplifies to $E(x_i^2)$.

Uniform integrability is usually invoked to shut down counterexamples like this one: if $x_i=i$ w.p. $1/i$ and zero w.p. $1-1/i$, we have $x_i\to_p0$ but $E(x_i)=1$ for any $i$, so expectations do not converge (to the plim). Also, $E(x_i^2)=i$, which diverges. Then, $$ E(Z_n)=\frac{n(n+1)}{2n}\to\infty $$ This, however, is evidently not an i.i.d. setup.

Davidson, Stochastic Limit Theory, Theorem 12.11, for example provides sufficient conditions for uniform integrability, such as uniformly bounded moments. That is $\{X_t\}$ is uniformly integrable if $E|X_t|^{1+\theta}<\infty$ for $\theta>0$.

  • $\begingroup$ Interesting post. Yes I noticed that the examples for when convergence in probability doesn't match convergence in expectation seem to always make a random variable $X_n$ under consideration heavily dependent on the index $n$ of the sequence (such as the image on the Wikipedia page. But for 'conventional' multiple regression in the real world an observation isn't dependent on the particular index $n$ of the sequence of observations. $\endgroup$ – sonicboom Dec 10 '20 at 17:24
  • $\begingroup$ For example if we regress heights against weights for all people in a particular city on a particular day. Then the rows of the data matrix $\mathbf{X}_n$ will feature observations of individual people's weights, and these will not be dependent on $n$. They are simply i.i.d observations. Its for this type of reason that I wonder if the $\{Y_n\}_{n=1}^\infty$ in my original question could be automatically uniformly integrable due to the i.i.d. assumption. $\endgroup$ – sonicboom Dec 10 '20 at 17:28
  • 2
    $\begingroup$ Indeed, as I was trying to express (not very well, apparently), I could not come up with an issue when there is iid-ness, and looking at the definition of uniform integrability suggests that it is relevant basically when there indeed is heterogeneity. $\endgroup$ – Christoph Hanck Dec 10 '20 at 18:17
  • $\begingroup$ Section 2 of this paper seems to imply the OLS estimators are uniformly integrable which I assume would mean $\left\{(\mathbf{X_n}^T\mathbf{X_n}/n)^{-1}\right\}_{n=1}^\infty$ is UI since $(\mathbf{X_n}^T\mathbf{X_n})^{-1}$ is used in the construction of the OLS estimators. $\endgroup$ – sonicboom Dec 13 '20 at 16:29
  • $\begingroup$ Actually I think that paper uses fixed regressors not stochastic regressors so it does not correspond to the case I am asking about. $\endgroup$ – sonicboom Mar 15 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.