0
$\begingroup$

I am not sure where to start with this. But I know for $Y_i = \beta_0+\beta_1X_i+\epsilon$, where $\beta_0,\beta_1,X$ are assumed to be constants and $\hat{Y_i} = b_0+b_1X_i$ is the simple linear regression model, where $b_0, b_1$ are random variables (or distributions under circumstance of repeated sampling).

Could anyone help show this?

$\endgroup$
1
  • $\begingroup$ $b_0$ and $b_1$ are the least-squares estimates, right? And $\epsilon$ is assumed independent of $X$? $\endgroup$ Mar 11, 2020 at 5:46

3 Answers 3

0
$\begingroup$

What is stated in the question is incorrect. Let us generalize, and use the matrix formulation of the linear model $Y=X\beta + \epsilon$ with $\DeclareMathOperator{\V}{\mathbb{V}} \V \epsilon=\sigma^2 I_n$, $X$ an $n\times p$ matrix of regressors, of ful column rank. Then as is known the least squares estimator is $\hat{\beta}= (X^TX)^{-1} X^T Y$:
$$ \DeclareMathOperator{\C}{\mathbb{C}} \begin{align} \C(\epsilon, \hat{\beta}) &=\C(\epsilon, (X^TX)^{-1} X^T Y) \\ &= \C(\epsilon,Y)X (X^TX)^{-1} \\ &= \C(\epsilon,X\beta+\epsilon) X (X^TX)^{-1} \\ &= \C(\epsilon,\epsilon) X (X^TX)^{-1} \\ &= \sigma^2 I_n X (X^TX)^{-1} = \sigma^2 X (X^TX)^{-1} \end{align} $$

$\endgroup$
0
$\begingroup$

The premise in the title of the question isn't correct. The estimated $b_0$ and $b_1$ are by definition functions of the observed data, which are functions of the $\epsilon_i$. Unless the question is misstated...

$\endgroup$
0
$\begingroup$

Both questions are bit confusing. For example, think of the covariance between e and b0: e varies across "people" and b0 (i.e., the sample estimate of the true unknown value of b0) varies across samples taken from a population. Therefore, it is strange to put it like you did. More meaningful would be, to ask e.g. why the covariance between the sample mean of all your sample e-values and the sample estimate of b0 equals zero.

In least squares, the e values are estimated in such a way that they sum to zero (in the sample) and that there covariance with any of the x variables is zero too (in the sample). These two things being zero follows from taking the first derivatives of the residual sum of squares and setting it equal to zero.

Regards, Ben.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.