1
$\begingroup$

First I need to minimize this just like I do with a regular L-S estimates (assuming Gauss-Markov) expect $\sum_{i=1}^n X_i=0$:

$$ \sum_i^n e_i^2=\sum_i^n(y_i-\beta_0+\beta_1x_i)^2 $$

I take the partial deriviates:

$$ -2\sum_i^n(y_i-\beta_0-\beta_1x_i) $$

$$ -2\sum_i^n(y_i-\beta_0-\beta_1x_i)x_i $$

Set equal to zero:

$$ \sum_i^n y_i-n\beta_0-\beta_1\sum_i^n x_i=0 \rightarrow \sum_i^n y_i-n\beta_0=0 \rightarrow n\beta_0=\sum_i^n y_i \rightarrow \beta_0=\bar{y} $$

$$ \sum_i^n x_iy_i-\beta_0x_i-\beta_1x_i^2=0 \rightarrow \beta_1\sum_i^nx_i^2=\sum_i^n x_iy_i \rightarrow \beta_1=\sum_i^n\frac{x_iy_i}{x_i^2}=\sum_i^n\frac{y_i}{x_i} $$

The expected values for these estimates are:

$$ E(\beta_0)=E(\bar(y))=E(\frac{\sum_i^n y_i}{n})=\frac{\sum_i^n E(y_i)}{n}=\frac{\sum_i^n \beta_0+\beta_1 \sum_i^nx_i}{n}=\beta_0+0=\beta_0 $$

$$ var(\beta_0)=var(\bar(y))=var(\frac{\sum_i^n y_i}{n})=\sum_i^n var(\frac{y_i}{n})=\sum_i^n \frac{var(y_i)}{n^2} = \frac{\sigma^2}{n} $$

I think my $\beta_0$ estimates are correct, but I feel like I did something wrong with $\beta_1$

I'd appreciate some guidance

$\endgroup$
4
  • 3
    $\begingroup$ The last equality for $\beta_1$ does not simplify like that. Be careful with the summation rules. $\endgroup$
    – JohnK
    Sep 20, 2015 at 8:39
  • $\begingroup$ Although your question refers to a "change," what is your reference? To what are you comparing these estimates? $\endgroup$
    – whuber
    Sep 20, 2015 at 13:10
  • $\begingroup$ I'm comparing to a regular L-S estimate where you don't assume $\sum_i^n X_i=0$ $\endgroup$ Sep 20, 2015 at 14:04
  • 1
    $\begingroup$ I still don't understand, because to make least-squares fits one knows the values of the $x_i$. Could you explain what it would mean, then, to "assume" the $X_i$ sum to zero? Are you supposing that the $x_i$ are iid draws from some zero-mean distribution? $\endgroup$
    – whuber
    Sep 21, 2015 at 14:35

1 Answer 1

1
$\begingroup$

Thanks I believe I got it.

$$ \sum_i^n x_iy_i-\beta_0x_i-\beta_1x_i^2=0 \rightarrow \beta_1x_i^2=\sum_i^n x_iy_i \rightarrow \beta_1=\frac{\sum_i^n x_iy_i}{\sum_i^n x_i^2} $$

$$ E(\beta_1)=\frac{\sum_i^n x_iE(y_i)}{\sum_i^n x_i^2}=\frac{\sum_i^n x_i(\beta_0+\beta_1x_i)}{\sum_i^n x_i^2}=\frac{\beta_0\sum_i^n x_i}{\sum_i^n x_i^2}+\frac{\beta_1\sum_i^n x_i^2}{\sum_i^n x_i^2}=\beta_1 $$

$$ var(\beta_1)=\sum_i^n var(c_iy_i)=\sum c_i^2\sigma^2=\sigma^2 \frac{\sum_i^n x_i^2}{(\sum_i^n x_i^2)^2}=\frac{\sigma^2}{\sum_i^n x_i^2} $$

$\endgroup$
1
  • 1
    $\begingroup$ Another way of stating the answer to the question in the post is that setting $x^*_i =x_i -\bar{x}$ orthogonalizes the intercept term $\beta_0$ from the remaining coefficients $\beta_i$ for $i=1,2,3 \dots, p$. $\endgroup$ Jan 12, 2018 at 3:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.