0
$\begingroup$

On an assignment I've been tasked with finding whether $$\hat{\beta}_1=\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2},$$ the estimator for the slope of a no-intercept regression model $Y_i=\beta_1 X_i+\epsilon_i$, is unbiased. I'm not sure how this is done. I would like to use the traditional expected value definition $\mathbb{E}(\hat{A})=A$, but I encounter trouble at the beginning with

$$\mathbb{E}\left(\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2}\right),$$

given that (at least from what I've seen),

$$\mathbb{E}\left(\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2}\right) \neq \frac{\mathbb{E}\left(\sum_{i=1}^n x_iy_i\right)}{\mathbb{E}(\sum_{i=1}^n x_i^2)}.$$

As such, I'm stuck. We haven't learned about the Gauss-Markov theorem (so this answer couldn't be used here) or Jensen's inequality, and we are not given the distribution of $X_i$ nor told to treat $\sum_{i=1}^n x_i^2$ as constant.

Any help would be greatly appreciated.

$\endgroup$

4 Answers 4

5
$\begingroup$

It does not matter whether the $X_i$ are random or fixed (so long as the probability they are all zero is $0$).

$$\mathbb E \left[ \frac{\sum X_iY_i}{\sum X_i^2}\right] = \mathbb E \left[ \frac{\sum bX_i^2 +X_i \epsilon_i}{\sum X_i^2}\right]= \mathbb E \left[b\frac {\sum X_i^2}{\sum X_i^2}\right] + \sum\mathbb E\left[\frac{ X_i }{\sum X_i^2}\epsilon_i\right] \\=b+0=b$$ using the independence of the $\epsilon_i$ from the $X_i$ and $\mathbb E \left[ \epsilon_i\right]=0$.

$\endgroup$
1
  • $\begingroup$ Thanks very much for the help. Clearly I faced the classic beginner mixup between $X_i$ and $x_i$... $\endgroup$
    – Kman3
    Sep 12, 2023 at 17:18
2
$\begingroup$

You don't need these theorems. $x_i$ is deterministic, and we have $y_i=\beta_1x_i+\epsilon_i$. So just substitute $y_i$ in the numerator with the above formula. Now just take expectation. The key is to just note that $x_i$ and $y_i$ are deterministic values and only $\epsilon_i$'s are random variables.

$\endgroup$
2
$\begingroup$

A major point is that the $x_i$ are NOT RANDOM and the $Y_i$ are random. (Note that below, I do not confuse $Y_i$ with $y_i$. The latter is an observed value; the former is a random variable.) So \begin{align} \operatorname E\left( \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2} \right) & = \frac{\operatorname E\left( \sum_{i=1}^n x_i Y_i \right)}{\sum_{i=1}^n x_i^2} \text{ because the $x$s are not random} \\[10pt] & = \frac{\sum_{i=1}^n \operatorname E(x_iY_i) }{\sum_{i=1}^n x_i^2} \text{ by linearity of expectation} \\[10pt] & = \frac{\sum_{i=1}^n x_i\operatorname E(Y_i)}{\sum_{i=1}^n x_i^2} \text{ because the $x$s are not random} \\[10pt] & = \frac{\sum_{i=1}^n x_i \big( \beta_1 x_i\big)}{\sum_{i=1}^n x_i^2} \\[10pt] & = \frac{\beta_1 \sum_{i=1}^n x_i \big( x_i\big)}{\sum_{i=1}^n x_i^2} \Large{ \begin{smallmatrix} \text{because $\beta_1$ does not} \\ \text{change as $i$ goes from $1$ to $n$} \end{smallmatrix}} \\[10pt] & = \beta_1. \end{align}

$\endgroup$
0
1
$\begingroup$

Explanatory variables $x_i$s are assumed to be known, fixed values. The randomness emanates from $\boldsymbol\varepsilon.$ Any linear estimator would be of the form $\mathbf c^\top\mathbf Y= \sum_i c_iY_i.$

If $\mathbb E[\sum_i c_iY_i]=\beta_1,$ this means $\sum c_ix_i=1.$ Imposing additional constraints and requirements for the estimator (viz. BLUE) would lead to particular value of $\mathbf c. $

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.