How to Prove Unbiased Estimator

Question

I'm unsure of how to convince myself that

$$\hat{\beta} = \frac{\sum X_i Y_i}{\sum X_i^2}$$

is an unbiased estimator when the regression model

$$Y_i = \beta X_i + \epsilon_i$$

follows basic OLS assumptions. To show this is unbiased, we need to show that $\text{E}( \hat{\beta} ) = \beta$.

My hunch is that the $X_i$ and $X_i^2$ will cancel out to give $\frac{Y_i}{X_i}$ (which is what I think $\beta$ equals?, but I'm not sure how to show it with the expectation). I get stuck with the $\text{E}(\sum X_i^2)$. My understanding is that this should equal $\text{Var}(X) \cdot \bar{X}^2$ This only gives $\text{E}(\hat{\beta}) = \beta$ if $\text{Var}(X) = 1$.

Answer 1

The key is that we are conditioning on the predictors $\{x_1, x_2, \ldots, x_n \}$ so they're viewed as constants. Once we realize this it becomes very straightforward:

\begin{align} \text{E} \left ( \frac{\sum_i x_i Y_i}{\sum_j x_j^2} \right ) &= \frac{ \sum_i x_i \text{E}(Y_i)}{\sum_j x_j^2} \\ &= \frac{ \beta \sum_i x_i^2}{\sum_j x_j^2} \\ &= \beta \end{align}

where we've used the fact that $\text{E}(Y_i) = \text{E}(\beta x_i + \epsilon_i) = \beta x_i$ since $\text{E}(\epsilon_i) = 0$.

Answer 2

The first thing to point out, is that your target equation

$$ \hat \beta = \beta $$

cannot be correct. The left hand side is a random variable, and the right hand side is a constant, so there is no hope to prove them equal, no matter how ingenious the algebra.

What you really want for unbiasedness is to show this

$$ E[\hat \beta \mid X] = \beta $$

Keeping in mind that conditioning allows you to treat $X$ as a constant, you should be able to get it from there.

Answer 3

Taking a look given below might solve your query.

https://www.youtube.com/watch?v=PriultFg8Qo