7
$\begingroup$

I do not understand why the following equality holds (taken from Cameron & Trivedi 2005: Microeconomtrics):

$\hat{\underline{\beta}}_{OLS}=(\textbf{X}'\textbf{X})^{-1}\textbf{X}'\textbf{y}=(\sum_{i=1}^{n}\textbf{x}_i\textbf{x}_{i}^{'})^{-1}\sum_{i=1}^{n}\textbf{x}_iy_i$

The notation is

$ \textbf{x}_{i}^{'} =(x_{i1},...,x_{ik})$

where $i$ designates the different observations of $k$ variables and $\textbf{X}$ stacks these upon another.

I have tried to develop the RHS of the equality, but I keep ending up with a scalar instead of a vector of coefficients.

Any help would be greatly appreciated!

$\endgroup$
8
$\begingroup$

$\textbf{X}$ is of dimension $\textbf{N}\times\textbf{K}$ and $\textbf{X}'$ of dimension $\textbf{K}\times\textbf{N}$, the product $(\textbf{X}'\textbf{X})$ is consequently of dimension $\textbf{K}\times\textbf{K}$.

Taking the invers of $\textbf{N}\times\textbf{N}$ does not change the dimension of the matrix. Consequently the multiplication of $(\textbf{X}'\textbf{X})\textbf{X}'$ is a multiplication of a matrix of dimension $\textbf{K}\times\textbf{K}$ with a matrix of dimension $\textbf{K}\times\textbf{N}$. There product is a matrix of dimension $\textbf{K}\times\textbf{N}$.

$\textbf{y}$ is a vector of $\textbf{N}\times1$. Multiplying $(\textbf{X}'\textbf{X})\textbf{X}'$, which is of dimension $\textbf{K}\times\textbf{N}$ with the vector $\textbf{y}$, which is of dimension $\textbf{N}\times1$ gives us a vector of $\textbf{K}\times1$ dimension.

I hope this sheds some light on you problem. Otherwise you should check out how the least squares estimator is derived. An explanation in a simple way can be found under http://economictheoryblog.com/2015/02/19/ols_estimator/

$\endgroup$
2
$\begingroup$

It appears that you're simply confused by the notation.

Here ${\bf x}_{i}$ is the column vector

${\bf x}_{i} = \left[\begin{array}{c} x_{i,1} \\ x_{i,2} \\ \vdots \\ x_{i,n} \end{array} \right].$

It is somewhat confusing that the author has chosen to take the $i$th row of the matrix ${\bf X}$ and call its transpose ${\bf x}_{i}$.

Next, when you compute the outer product ${\bf x}_{i}{\bf x}_{i}'$, you get an $n$ by $n$ matrix:

${\bf x}_{i}{\bf x}_{i}'=\left[\begin{array}{ccccc} {\it x}_{i,1}x_{i,1} & x_{i,1}x_{i,2} & x_{i,1}x_{i,3} & \ldots & x_{i,1}x_{i,n} \\ x_{i,2}x_{i,1} & x_{i,2}x_{i,2} & x_{i,2}x_{i,3} & \ldots & x_{i,2}x_{i,n} \\ \vdots & \vdots & \vdots & \vdots & \vdots \\ x_{i,n}x_{i,1} & x_{i,n}x_{i,2} & x_{i,n}x_{i,3} & \ldots & x_{i,n}x_{i,n} \\ \end{array} \right].$

Finally, you need to be aware of the sum of outer products form of matrix multiplication, which gives

${\bf X}'{\bf X}=\sum_{i=1}^{n}{\bf x}_{i}{\bf x}_{i}'.$

$\endgroup$
  • $\begingroup$ This is "standard confusion" in econometrics! $\endgroup$ – Alecos Papadopoulos May 13 '14 at 16:20
0
$\begingroup$

You are referring to Cameron & Trivedi, page 16, aren't you? There they say that "vectors are defined as column vectors". This is why in $y_i=\mathbf{x}_i'\boldsymbol{\beta}+u_i$ (same page) $\mathbf{x}_i$ is the column vector that contains... the $i$th row of the $N\times K$ $\mathbf{X}$ matrix.

So $\mathbf{x}_i'\boldsymbol{\beta}$ is a scalar, $\mathbf{x}_i\mathbf{x}_i'$ is a matrix.

Wooldridge writes $y_i=\mathbf{x}_i\boldsymbol{\beta}$ and $\sum_{i=1}^N\mathbf{x}_i'\mathbf{x}_i$. It's just a different convention.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.