# OLS assumption, full rank of matrix $X$

One of the OLS assumptions concerning the $$X$$-matrix (with a constant) is that the columns $$(1, x_{i1}, \ldots , x_{iK})$$ are not linearly dependent. This looks intuitive to me, because of the dummy-variable trap.

In another text book I found the assumption is $$E[x_i x_i']$$ is positive definite and finite and that the $$\text{rank}(X) = K + 1$$.

Now I understand that $$X$$ has to be a full rank matrix, because the column rank is equal to its row rank. So that is the same statement as in the first assumption. But how is "$$E[x_i x_i']$$ is positive definite and finite" related to that?

They are closely related, but not the same. The first relates to the actual regressors observed, whereas the second to the population from which we draw. When a LLN works, that will amount to the same thing in large samples (that is at least my intuition, I would not rule out maybe somewhat pathological counterexamples), but not necessarily in small samples.

Consider a regression model with $$K=2$$ dummy regressors $$x_{1i}$$ (say, rural/urban) and $$x_{2i}$$ (say, employed/not employed), $$y_i=\beta_0+\beta_1x_{1i}+\beta_2x_{2i}+u_i$$ Assume $$P(x_{1i}=1)=p_1$$ and $$P(x_{2i}=1)=p_2$$ and, for simplicity (that is not necessary for the argument) that the two regressors are independent. Then, $$E(x_ix_i')=\begin{pmatrix}1&p_1&p_2\\p_1&p_1&p_1p_2\\p_2&p_1p_2&p_2\end{pmatrix}$$ will have full rank $$K+1=3$$ (barring the pathological case when $$p_j\in\{0,1\}$$, $$j=1,2$$---the determinant is $$(p_1 - 1) p_1 (p_2 - 1) p_2$$).

In turn, that the actual observed regression matrix $$X$$ does not have full rank can, in particular in small samples, still occur.

Try, e.g.,

set.seed(13)
n <- 4
p1 <- 0.3
p2 <- 0.2
X <- cbind(1, rbinom(n, 1, p1), rbinom(n, 1, p2))

> X
[,1] [,2] [,3]
[1,]    1    1    1
[2,]    1    0    0
[3,]    1    0    0
[4,]    1    0    0

> qr(X)\$rank
[1] 2


For the "converse case" showing that we also cannot conclude from full rank for any $$n$$ to identification (see D F's +1 answer), consider $$y_t=\beta \frac{1}{t}+u_t,$$ i.e., a deterministic regressor $$x_t=1/t$$ that tends to zero as $$t\to\infty$$. Here, the "expectation" hence is actually just $$1/t$$ itself and therefore also not constant.

In such cases, we have heterogeneously distributed data (I do not know if that is necessary for such a converse case) and need to resort to limit theorems such as Exercise 3.14 (for consistency) or 5.12 (for asymptotic normality) in White, Asymptotic Theory for Econometricians, that requires the "average" variance matrix of the regressors

$$M_n\equiv E(X'X/n)$$ to be uniformly positive definite

Here, $$E(X'X/n)=\frac{X'X}{n}=\frac{1}{n}\sum_{t=1}^n\frac{1}{t^2}$$ This quantity clearly is positive for any $$n$$, but, since $$\sum_{t=1}^n\frac{1}{t^2}\to\frac{\pi^2}{6},$$ as $$n\to\infty$$, we have that $$\frac{1}{n}\sum_{t=1}^n\frac{1}{t^2}\to 0$$ as $$n\to\infty$$. Hence, the condition asymptotically fails in this case.

As for the requirement for the variance matrix to be finite, you could turn this example on its head and consider a trend regressor $$\beta t$$ whose sum of squares obviously diverges. (This is not to say that regressions with such regressors do not "work", but that they are not covered by the present conditions.)

The two assumptions you're referring to are a bit different. They pertain to identification and estimation that are conceptually different issues. Before estimating the linear model $$y = x' \beta + u$$ you first need to make sure that the parameter of interest is identifiable from the data. Which can informally be rephrased as the parameter of interest ($$\beta$$) can be expressed uniquely as a function of the joint distribution of $$(Y, X)$$. To guarantee this you need to assume that the matrix $$E[xx']$$ is finite and invertible (since $$E[xx']$$ is always non-negative semidefinite, to guarantee invertibility you need to assume that it is in fact positive definite).

But then, when you go on to estimation, the OLS estimator is $$\hat{\beta}^{OLS} = (X'X)^{-1}X'Y$$ to make sure that this estimator exists you need to make sure that the matrix $$X'X$$ is invertible. A necessary and sufficient condition for this is the rank condition $$rk(X) = k+1$$ (or alternatively, linear independence of columns).

Notice that neither of these two assumptions implies the other (so they are indeed different). However, if $$E[xx']$$ is invertible then as $$n$$ gets large $$X'X$$ will be almost surely invertible.