0
$\begingroup$

For a regression line through the origin with the equation:

$$ \tilde{y}=\tilde{\beta_1}x $$

How did we use OLS to get the below equation? I know it is by minimising the SSR but I can't seem to work it out by plugging in the values into the formula for SSR.

$$ \sum (y_i -\tilde{\beta_1}x_i)^2 $$

And furthermore, how do we use calculus to get the first order condition for equation directly above?

First order condition:

$$ \sum x_i(y_i-\tilde{\beta_1}x_i) = 0 $$

Is it a partial derivative? If so, where did the exponent (2) go?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This is just the derivative. For example, our loss function is the sum of squared residuals, or $$ S(\beta) = \sum_{i=1}^n (y_i - \beta x_i)^2. $$ We want to minimize this, so we take the first derivative: $$ \frac{dS}{d\beta} = -2 \sum_{i=1}^n (y_i - \beta x_i) x_i. $$

We set it equal to zero to find the minimum, so $$ -2 \sum_{i=1}^n (y_i - \hat{\beta} x_i) x_i = 0 \\ \hat{\beta} =\frac{\sum_{i=1}^n y_i x_i}{\sum_{i=1}^n x_i^2}. $$

The exponent gets pulled down (it's the power rule in calculus).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Why is our loss function the sum of squared residuals? Is it because $ \hat{u}=\tilde{y}-\tilde{beta_1}x $? $\endgroup$
    – Munir Malik
    Nov 20, 2019 at 19:47
  • $\begingroup$ For OLS (ordinary least squares), the loss function is the sum of squared errors, as it yields estimators with good properties. You could certainly use a different loss function on a regression problem, but then it wouldn't be OLS. $\endgroup$
    – David Atlas
    Nov 20, 2019 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.