0
$\begingroup$

For a regression line through the origin with the equation:

$$ \tilde{y}=\tilde{\beta_1}x $$

How did we use OLS to get the below equation? I know it is by minimising the SSR but I can't seem to work it out by plugging in the values into the formula for SSR.

$$ \sum (y_i -\tilde{\beta_1}x_i)^2 $$

And furthermore, how do we use calculus to get the first order condition for equation directly above?

First order condition:

$$ \sum x_i(y_i-\tilde{\beta_1}x_i) = 0 $$

Is it a partial derivative? If so, where did the exponent (2) go?

$\endgroup$
0
$\begingroup$

This is just the derivative. For example, our loss function is the sum of squared residuals, or $$ S(\beta) = \sum_{i=1}^n (y_i - \beta x_i)^2. $$ We want to minimize this, so we take the first derivative: $$ \frac{dS}{d\beta} = -2 \sum_{i=1}^n (y_i - \beta x_i) x_i. $$

We set it equal to zero to find the minimum, so $$ -2 \sum_{i=1}^n (y_i - \hat{\beta} x_i) x_i = 0 \\ \hat{\beta} =\frac{\sum_{i=1}^n y_i x_i}{\sum_{i=1}^n x_i^2}. $$

The exponent gets pulled down (it's the power rule in calculus).

$\endgroup$
  • $\begingroup$ Why is our loss function the sum of squared residuals? Is it because $ \hat{u}=\tilde{y}-\tilde{beta_1}x $? $\endgroup$ – Munir Malik Nov 20 '19 at 19:47
  • $\begingroup$ For OLS (ordinary least squares), the loss function is the sum of squared errors, as it yields estimators with good properties. You could certainly use a different loss function on a regression problem, but then it wouldn't be OLS. $\endgroup$ – David Atlas Nov 20 '19 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.