1
$\begingroup$

First I need to minimize this just like I do with a regular L-S estimates (assuming Gauss-Markov) expect $\sum_{i=1}^n X_i=0$:

$$ \sum_i^n e_i^2=\sum_i^n(y_i-\beta_0+\beta_1x_i)^2 $$

I take the partial deriviates:

$$ -2\sum_i^n(y_i-\beta_0-\beta_1x_i) $$

$$ -2\sum_i^n(y_i-\beta_0-\beta_1x_i)x_i $$

Set equal to zero:

$$ \sum_i^n y_i-n\beta_0-\beta_1\sum_i^n x_i=0 \rightarrow \sum_i^n y_i-n\beta_0=0 \rightarrow n\beta_0=\sum_i^n y_i \rightarrow \beta_0=\bar{y} $$

$$ \sum_i^n x_iy_i-\beta_0x_i-\beta_1x_i^2=0 \rightarrow \beta_1\sum_i^nx_i^2=\sum_i^n x_iy_i \rightarrow \beta_1=\sum_i^n\frac{x_iy_i}{x_i^2}=\sum_i^n\frac{y_i}{x_i} $$

The expected values for these estimates are:

$$ E(\beta_0)=E(\bar(y))=E(\frac{\sum_i^n y_i}{n})=\frac{\sum_i^n E(y_i)}{n}=\frac{\sum_i^n \beta_0+\beta_1 \sum_i^nx_i}{n}=\beta_0+0=\beta_0 $$

$$ var(\beta_0)=var(\bar(y))=var(\frac{\sum_i^n y_i}{n})=\sum_i^n var(\frac{y_i}{n})=\sum_i^n \frac{var(y_i)}{n^2} = \frac{\sigma^2}{n} $$

I think my $\beta_0$ estimates are correct, but I feel like I did something wrong with $\beta_1$

I'd appreciate some guidance

$\endgroup$
  • 2
    $\begingroup$ The last equality for $\beta_1$ does not simplify like that. Be careful with the summation rules. $\endgroup$ – JohnK Sep 20 '15 at 8:39
  • $\begingroup$ Although your question refers to a "change," what is your reference? To what are you comparing these estimates? $\endgroup$ – whuber Sep 20 '15 at 13:10
  • $\begingroup$ I'm comparing to a regular L-S estimate where you don't assume $\sum_i^n X_i=0$ $\endgroup$ – jimduquettesucked Sep 20 '15 at 14:04
  • $\begingroup$ I still don't understand, because to make least-squares fits one knows the values of the $x_i$. Could you explain what it would mean, then, to "assume" the $X_i$ sum to zero? Are you supposing that the $x_i$ are iid draws from some zero-mean distribution? $\endgroup$ – whuber Sep 21 '15 at 14:35
0
$\begingroup$

Thanks I believe I got it.

$$ \sum_i^n x_iy_i-\beta_0x_i-\beta_1x_i^2=0 \rightarrow \beta_1x_i^2=\sum_i^n x_iy_i \rightarrow \beta_1=\frac{\sum_i^n x_iy_i}{\sum_i^n x_i^2} $$

$$ E(\beta_1)=\frac{\sum_i^n x_iE(y_i)}{\sum_i^n x_i^2}=\frac{\sum_i^n x_i(\beta_0+\beta_1x_i)}{\sum_i^n x_i^2}=\frac{\beta_0\sum_i^n x_i}{\sum_i^n x_i^2}+\frac{\beta_1\sum_i^n x_i^2}{\sum_i^n x_i^2}=\beta_1 $$

$$ var(\beta_1)=\sum_i^n var(c_iy_i)=\sum c_i^2\sigma^2=\sigma^2 \frac{\sum_i^n x_i^2}{(\sum_i^n x_i^2)^2}=\frac{\sigma^2}{\sum_i^n x_i^2} $$

$\endgroup$
  • $\begingroup$ Another way of stating the answer to the question in the post is that setting $x^*_i =x_i -\bar{x}$ orthogonalizes the intercept term $\beta_0$ from the remaining coefficients $\beta_i$ for $i=1,2,3 \dots, p$. $\endgroup$ – Lucas Roberts Jan 12 '18 at 3:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.