# Question about expectation in OLS?

Consider the linear model $$y_i = x_i^T\beta + \epsilon_i.$$

In ordinary least squares it is assumed that the errors satisfy $$E[\epsilon_i]=0$$. This implies that that $$\dfrac{X^T\epsilon}{n} \to 0$$ in probability, where $$X$$ is the design matrix and it is non-stochastic. What is the reasoning behind this claim?

That result is not implied by the distribution of the error terms. Asymptotic convergence results in regression analysis ---which treat the explanatory variables as fixed--- require a separate assumption that ensure that the explanatory variables do not grow too rapidly in magnitude. In the case of IID error terms with $$\mathbb{E}(\epsilon_i) = 0$$ and $$\mathbb{V}(\epsilon_i) = \sigma^2$$ you have:

$$\bigg( \frac{\mathbf{x}_n \cdot \boldsymbol{\epsilon}_n}{n} \bigg)^2 \leqslant \frac{||\mathbf{x}_n||^2 \cdot ||\boldsymbol{\epsilon_n}||^2}{n^2} = \frac{||\mathbf{x}_n||^2}{n} \cdot \frac{||\boldsymbol{\epsilon}_n||^2}{n} \rightarrow \frac{\sum_{i=1}^n x_i^2}{n} \cdot \sigma^2.$$

Thus, a sufficient condition for the convergence you are seeking is to assume that $$\sum_{i=1}^n x_i^2/n \rightarrow 0$$.

• This condition appears to be unnecessarily strong. – Alecos Papadopoulos Feb 14 at 2:51

We essentially examine the conditions under which the Law of Large Numbers holds for the sum

$$\frac 1n \sum_{i=1}^nx_{ki}u_i,\;\;\; E(u_i)=0$$

for every $$k=1,...K$$ regressor, and we assume also a finite variance for $$u_i$$.

Now, when the $$x_{ki}$$'s are non-stochastic then they are just a sequence of real numbers, and we might as well re-write the sum as

$$\frac 1n \sum_{i=1}^na_{ki}u_i,\;\;\; E(u_i)=0$$

I write it like this to stress that the only source of randomness here is the $$u_i$$'s, and so that, what we are looking at is the average of independent but non-identically distributed random variables $$z_{ki}=a_{ki}u_i$$. This is the "Chebyshev's" Law of Large Numbers and requires that the variance of each random variable is finite. This means that we need $$\text{Var}(z_{ki}) = a_{ki}^2 \text{Var}(u_i) < \infty,\;\;\; \forall i$$

Markov generalized this LLN to possibly non-independent random variables, where we require that

$$\text{Var}\left(\frac 1n \sum_{i=1}^na_{ki}u_i\right) \to 0$$

Under independence of $$u_i$$'s, covariances are zero and we have

$$\text{Var}\left(\frac 1n \sum_{i=1}^na_{ki}u_i\right) = \text{Var}(u_i)\cdot \frac 1{n^2}\sum_{i=1}^na_{ki}^2$$ and so the sufficient condition we need is

$$\frac 1{n^2}\sum_{i=1}^na_{ki}^2 \to 0$$

or to revert back to original notation

$$\frac 1{n^2}\sum_{i=1}^nx_{ki}^2 \to 0$$