Trying to show $E[\hat \beta_1 | \mathbf{X}] = \beta_1$ directly from the definition of $\hat \beta_1$?

Question

Suppose we have the standard simple linear regression model: $$ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i, $$ with $E[\varepsilon_i|X_i] = 0$ and $\text{Var}[\varepsilon_i|X_i] = \sigma^2$.

I'm trying to show that $$ E[\hat \beta_1 | \mathbf{X}] = \beta_1, $$ directly using the definition of $\hat \beta_1$, where $\mathbf{X}$ is the vector of $X_i$'s. I know there are other ways to show it but I am trying to do it this way so that I can practise working with conditional expectation. The definition of $\hat \beta_1$ is $$ \hat \beta_1 = \frac{\sum (X_i - \bar X)(Y_i - \bar Y)}{\sum (X_i - \bar X)^2}. $$

Define $$ g_i(\mathbf{X}) := \frac{X_i - \bar X}{\sum (X_i - \bar X)^2}. $$

Here's what I've done: $$ \begin{align} E[\hat \beta_1 | \mathbf{X}] & = E\bigg[\frac{\sum (X_i - \bar X)(Y_i - \bar Y)}{\sum (X_i - \bar X)^2} \bigg| \mathbf{X}\bigg] \\ & = E\bigg[\sum_i g_i(\mathbf{X})(Y_i - \bar Y) \bigg| \mathbf{X} \bigg] \\ & = \sum_i E\bigg[g_i(\mathbf{X})(Y_i - \bar Y) \bigg| \mathbf{X} \bigg] \\ & = \sum_i E[g_i(\mathbf{X})Y_i| \mathbf{X}] - \sum_i E[g_i(\mathbf{X}) \bar Y | \mathbf{X} ] \\ & = \sum_i g_i(\mathbf{X}) E[Y_i| \mathbf{X} ] - \sum_i g_i(\mathbf{X}) E[\bar Y | \mathbf{X}] \\ \end{align} $$ Because I can take the $g(\mathbf{X})$ out of the expectation it seems we can never get a constant $\beta_1$ as the final result? Where have I gone wrong? How can we show $E[\hat \beta_1 | \mathbf{X}] = \beta_1$ using this approach?

Answer 1

Following from the third line, $E[Y_i-\bar Y|\mathbf X]=(\beta_0+\beta_1X_i)-(\beta+\beta_1\bar X)=\beta_1(X_i-\bar X)$. When substituted that back, we have $$\begin{align}E[\hat \beta_1|\mathbf X]&=\sum_{i} \beta_1g_i(\mathbf X)(X_i-\bar X)=\beta_1\sum_i\frac{(X_i-\bar X)}{\sum_j (X_j-\bar X)^2}(X_i-\bar X)\\&=\beta_1\frac{\sum_i (X_i-\bar X)^2}{\sum_j (X_j-\bar X)^2}=\beta_1\end{align}$$

By the way, careful about the summation indices. $i$ in denominator expression is different from $i$ in the numerator. So, $$g_i(\mathbf X)=\frac{X_i-\bar X}{\sum_j(X_j-\bar X)^2}$$

Answer 2

You can see it as

$$ \hat{\beta}=(X^tX)^{-1}X^ty = (X^tX)^{-1}X^t(X\beta+\varepsilon) =(X^tX)^{-1}X^tX\beta + (X^tX)^{-1}X^t\varepsilon $$

So you have that

$$ \hat{\beta}=\beta + (X^tX)^{-1}X^t\varepsilon $$

And then, it is straightforward to see that

$$ \mathbb{E}(\hat\beta) = \mathbb{E}(\beta + (X^tX)^{-1}X^t\varepsilon) = \beta + (X^tX)^{-1}X^t\mathbb{E}(\varepsilon) $$

But $\mathbb{E}(\varepsilon)=0$

So $\hat\beta$ is unbiased