OLS Coefficient estimator; Transformation from Matrix to sum of matrices form

Question

I do not understand why the following equality holds (taken from Cameron & Trivedi 2005: Microeconomtrics):

$\hat{\underline{\beta}}_{OLS}=(\textbf{X}'\textbf{X})^{-1}\textbf{X}'\textbf{y}=(\sum_{i=1}^{n}\textbf{x}_i\textbf{x}_{i}^{'})^{-1}\sum_{i=1}^{n}\textbf{x}_iy_i$

The notation is

$ \textbf{x}_{i}^{'} =(x_{i1},...,x_{ik})$

where $i$ designates the different observations of $k$ variables and $\textbf{X}$ stacks these upon another.

I have tried to develop the RHS of the equality, but I keep ending up with a scalar instead of a vector of coefficients.

Any help would be greatly appreciated!

Answer 1

$\textbf{X}$ is of dimension $\textbf{N}\times\textbf{K}$ and $\textbf{X}'$ of dimension $\textbf{K}\times\textbf{N}$, the product $(\textbf{X}'\textbf{X})$ is consequently of dimension $\textbf{K}\times\textbf{K}$.

Taking the invers of $\textbf{N}\times\textbf{N}$ does not change the dimension of the matrix. Consequently the multiplication of $(\textbf{X}'\textbf{X})\textbf{X}'$ is a multiplication of a matrix of dimension $\textbf{K}\times\textbf{K}$ with a matrix of dimension $\textbf{K}\times\textbf{N}$. There product is a matrix of dimension $\textbf{K}\times\textbf{N}$.

$\textbf{y}$ is a vector of $\textbf{N}\times1$. Multiplying $(\textbf{X}'\textbf{X})\textbf{X}'$, which is of dimension $\textbf{K}\times\textbf{N}$ with the vector $\textbf{y}$, which is of dimension $\textbf{N}\times1$ gives us a vector of $\textbf{K}\times1$ dimension.

I hope this sheds some light on you problem. Otherwise you should check out how the least squares estimator is derived. An explanation in a simple way can be found under http://economictheoryblog.com/2015/02/19/ols_estimator/

Answer 2

It appears that you're simply confused by the notation.

Here ${\bf x}_{i}$ is the column vector

${\bf x}_{i} = \left[\begin{array}{c} x_{i,1} \\ x_{i,2} \\ \vdots \\ x_{i,n} \end{array} \right].$

It is somewhat confusing that the author has chosen to take the $i$th row of the matrix ${\bf X}$ and call its transpose ${\bf x}_{i}$.

Next, when you compute the outer product ${\bf x}_{i}{\bf x}_{i}'$, you get an $n$ by $n$ matrix:

${\bf x}_{i}{\bf x}_{i}'=\left[\begin{array}{ccccc} {\it x}_{i,1}x_{i,1} & x_{i,1}x_{i,2} & x_{i,1}x_{i,3} & \ldots & x_{i,1}x_{i,n} \\ x_{i,2}x_{i,1} & x_{i,2}x_{i,2} & x_{i,2}x_{i,3} & \ldots & x_{i,2}x_{i,n} \\ \vdots & \vdots & \vdots & \vdots & \vdots \\ x_{i,n}x_{i,1} & x_{i,n}x_{i,2} & x_{i,n}x_{i,3} & \ldots & x_{i,n}x_{i,n} \\ \end{array} \right].$

Finally, you need to be aware of the sum of outer products form of matrix multiplication, which gives

${\bf X}'{\bf X}=\sum_{i=1}^{n}{\bf x}_{i}{\bf x}_{i}'.$

Answer 3

You are referring to Cameron & Trivedi, page 16, aren't you? There they say that "vectors are defined as column vectors". This is why in $y_i=\mathbf{x}_i'\boldsymbol{\beta}+u_i$ (same page) $\mathbf{x}_i$ is the column vector that contains... the $i$th row of the $N\times K$ $\mathbf{X}$ matrix.

So $\mathbf{x}_i'\boldsymbol{\beta}$ is a scalar, $\mathbf{x}_i\mathbf{x}_i'$ is a matrix.

Wooldridge writes $y_i=\mathbf{x}_i\boldsymbol{\beta}$ and $\sum_{i=1}^N\mathbf{x}_i'\mathbf{x}_i$. It's just a different convention.