# Relationship between Linear Projection and OLS Regression

In Wooldridge's Econometric Analysis of Cross Section and Panel Data, he defines linear projection of $y$ on $1,\mathbf{x}$, in the following way:

Let's assume that $Var(\mathbf{x})$ is positive-definite, then the linear projection of $y$ on $\mathbf{x}$ exists and is unique such that

$L(y|1,\mathbf{x})=\beta_0+\mathbf{x}\beta$, where by definition

$\beta=(Var(\mathbf{x}))^{-1}Cov(\mathbf{x},y)$ and $\beta_0=E(y)-E(\mathbf{x})\beta$.

First I do not see how the author reaches that definition, and how it relates to this one in Wikipedia's article on projection. We usually have $P=X(X'X)^{-1}X'$, where $X$ is the matrix with columns $1,\text{and }\mathbf{x}$, for the sample (not sure how to write this projection w.r.t. the population though). Second, instead of defining the betas as above, couldn't we have derived those equations? How?

Any help would be appreciated.

Edit: According to the book, the notation is the following. $\mathbf{x}$ is a row vector of dimension $K$, so the dimension of the design matrix is $N\times K$

Another formulation of the $\beta$ regression parameter estimator is as

$\hat{\beta} = \left(\mathbf{X}^T\mathbf{X}\right)^{-1} \mathbf{X}^T Y$

Here $\hat{\beta}$ is a two element vector of $\hat{\beta}_0$ the intercept and $\hat{\beta}_1$ the slope. I like to use $\mathbf{X}$ notationally as a design matrix with the principal column a vector of 1s.

It's easy to see the crossproducts have a factor of $n$ that cancels out in these operations. WLOG we may assume that the random component(s) of $\mathbf{X}$ are centered, you have:

$\mbox{Cov} (\mathbf{X}) = \frac{1}{n}\left( \mathbf{X}^T \mathbf{X} \right)$

and

$\mbox{Cov} (\mathbf{X}, Y) = \frac{1}{n}\left( \mathbf{X}^T Y \right)$

Therefore the least squares estimator can be expressed as $\beta_1 = E(\hat{\beta}_1) = \mbox{Cov}(X, Y) / \mbox{Var} (X)$ for univariate models.

The projection matrix is formulated as $P = \mathbf{X}\left(\mathbf{X}^T\mathbf{X}\right)^{-1} \mathbf{X}^T$ and the predicted values of $Y$ (e.g. $\hat{Y}$) are given by:

$\hat{Y} = PY = \mathbf{X}\left(\mathbf{X}^T\mathbf{X}\right)^{-1} \mathbf{X}^TY = \mathbf{X} \hat{\beta}$

and you'll see it's a projection. All predicted values of $Y$ are formed using a linear combination of vectors of $\mathbf{X}$ e.g. they span the basis of $\mathbf{X}$. The projection specifically "projects" the values of $Y$ onto the fitted values $\hat{Y}$.

The author has formulated the fitted value or conditional mean function using unusual notation $L(y | 1, x)$ is basically equivalent to $\hat{Y}$

Alternately, the hat matrix, or influence matrix, is $H = \mathcal{I} - P$ and the residuals are given by $r = HY$.

Reference: Seber, Lee 2nd edition 1990.

• Thanks for your answer. I'm not sure about something on your answer. How would one write that but in terms of population moments? If the author was speaking of the estimate of betas, then I think he would have written so... – An old man in the sea. May 12 '16 at 7:46
• Also, according to the notation used $\mathbf{x}$ is not the design matrix, but just the row vector $(x_1,...,x_K)$ – An old man in the sea. May 12 '16 at 13:28
• @Anoldmaninthesea. can you clarify exactly what you're asking? I don't understand your first comment. On the second, I think it's relatively straightforward to see through any notational differences. You will need matrix notation for the design matrix to connect the estimation and definition of LS betas to projections as your question stated. – AdamO May 12 '16 at 16:16
• Adam, I'll try. In my 1st question, I'm asking why your using an estimate of beta. Usually, in the question's notation, $\beta= E(\mathbf{x}^T\mathbf{x})^{-1}E(\mathbf{x}^Ty)$. In this equation we use the population moments, and each row of the design matrix are draws from the population. – An old man in the sea. May 13 '16 at 17:57
• @Anoldmaninthesea. You've answered your own question, the estimator defines the parameter: $\beta_1 = \sigma_{xy} / \sigma_{x}^2 = E(\mbox{cov} (X, Y) / \mbox{var}(X))$. Usually we think of the design matrix $X$ as "fixed", though. We think of $X$ as coming from its empirical distribution so that $\mbox{var}{X} = \sum_i^n(X_i - \bar{X})^2 / {n} = n^{-1} \vec{X}^T\vec{X}$ for $X$ centered. Similarly, $\sigma_{x, y} = E(X^TY)$. The formulation you present is more probabilistic than I'd like. Statistically, it is more instructive to see that for a finite realization these properties hold. – AdamO May 13 '16 at 19:28