# Trying to show $E[\hat \beta_1 | \mathbf{X}] = \beta_1$ directly from the definition of $\hat \beta_1$?

Suppose we have the standard simple linear regression model: $$Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i,$$ with $$E[\varepsilon_i|X_i] = 0$$ and $$\text{Var}[\varepsilon_i|X_i] = \sigma^2$$.

I'm trying to show that $$E[\hat \beta_1 | \mathbf{X}] = \beta_1,$$ directly using the definition of $$\hat \beta_1$$, where $$\mathbf{X}$$ is the vector of $$X_i$$'s. I know there are other ways to show it but I am trying to do it this way so that I can practise working with conditional expectation. The definition of $$\hat \beta_1$$ is $$\hat \beta_1 = \frac{\sum (X_i - \bar X)(Y_i - \bar Y)}{\sum (X_i - \bar X)^2}.$$

Define $$g_i(\mathbf{X}) := \frac{X_i - \bar X}{\sum (X_i - \bar X)^2}.$$

Here's what I've done: \begin{align} E[\hat \beta_1 | \mathbf{X}] & = E\bigg[\frac{\sum (X_i - \bar X)(Y_i - \bar Y)}{\sum (X_i - \bar X)^2} \bigg| \mathbf{X}\bigg] \\ & = E\bigg[\sum_i g_i(\mathbf{X})(Y_i - \bar Y) \bigg| \mathbf{X} \bigg] \\ & = \sum_i E\bigg[g_i(\mathbf{X})(Y_i - \bar Y) \bigg| \mathbf{X} \bigg] \\ & = \sum_i E[g_i(\mathbf{X})Y_i| \mathbf{X}] - \sum_i E[g_i(\mathbf{X}) \bar Y | \mathbf{X} ] \\ & = \sum_i g_i(\mathbf{X}) E[Y_i| \mathbf{X} ] - \sum_i g_i(\mathbf{X}) E[\bar Y | \mathbf{X}] \\ \end{align} Because I can take the $$g(\mathbf{X})$$ out of the expectation it seems we can never get a constant $$\beta_1$$ as the final result? Where have I gone wrong? How can we show $$E[\hat \beta_1 | \mathbf{X}] = \beta_1$$ using this approach?

Following from the third line, $$E[Y_i-\bar Y|\mathbf X]=(\beta_0+\beta_1X_i)-(\beta+\beta_1\bar X)=\beta_1(X_i-\bar X)$$. When substituted that back, we have \begin{align}E[\hat \beta_1|\mathbf X]&=\sum_{i} \beta_1g_i(\mathbf X)(X_i-\bar X)=\beta_1\sum_i\frac{(X_i-\bar X)}{\sum_j (X_j-\bar X)^2}(X_i-\bar X)\\&=\beta_1\frac{\sum_i (X_i-\bar X)^2}{\sum_j (X_j-\bar X)^2}=\beta_1\end{align}

By the way, careful about the summation indices. $$i$$ in denominator expression is different from $$i$$ in the numerator. So, $$g_i(\mathbf X)=\frac{X_i-\bar X}{\sum_j(X_j-\bar X)^2}$$

You can see it as

$$\hat{\beta}=(X^tX)^{-1}X^ty = (X^tX)^{-1}X^t(X\beta+\varepsilon) =(X^tX)^{-1}X^tX\beta + (X^tX)^{-1}X^t\varepsilon$$

So you have that

$$\hat{\beta}=\beta + (X^tX)^{-1}X^t\varepsilon$$

And then, it is straightforward to see that

$$\mathbb{E}(\hat\beta) = \mathbb{E}(\beta + (X^tX)^{-1}X^t\varepsilon) = \beta + (X^tX)^{-1}X^t\mathbb{E}(\varepsilon)$$

But $$\mathbb{E}(\varepsilon)=0$$

So $$\hat\beta$$ is unbiased

• You appear to be treating $X$ as a constant, whereas the question concerns the case where it is a random variable.
– whuber
Oct 29, 2020 at 17:39
• Here I am making the common asupmtions that $X$ is a full-rank matrix of predictors, and that $\varepsilon\sim N(0, \sigma I)$. So, answering your question, I am treating X as a matrix of covariates and $\epsilon$ as a random variable. As I say, those are common assumptions. Oct 29, 2020 at 17:47
• Your mathematics does not reflect those assumptions, because under them $E(\hat\beta)$ should be a vector but you have obtained an expression in terms of $\beta,$ $E[\varepsilon],$ and $X$--which is a random variable.
– whuber
Oct 29, 2020 at 20:16