8
$\begingroup$

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1) \\ (5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3) $$ How can we find the best fitting line $y=ax$ through the points? My calculator has the option to find the best fitting line $y=ax+b$ through these points, which is:

$$y = 53.28x + 0.37$$

How can I find the best fitting $y=ax$? It seems to me we can't just remove the $0.37$ without compensating in the $a$?

$\endgroup$
  • 2
    $\begingroup$ Is there some reason why you want to? Suppressing the intercept leads to a biased model except if the intercept is exactly zero to infinite decimal places. Even then, you don't gain much efficiency. $\endgroup$ – gung Mar 31 '13 at 16:27
  • 1
    $\begingroup$ These are results of a physics experiment. If it has a y-intercept, It would lead to completely incorrect stuff. $\endgroup$ – EdwardHarrison Mar 31 '13 at 16:29
  • $\begingroup$ @gung Would that mean we just remove the $0.37$? $\endgroup$ – EdwardHarrison Mar 31 '13 at 16:30
  • 5
    $\begingroup$ "Suppressing the intercept" does not mean simply deleting the estimate from your model, it means fitting a model via a different formula that coerces the line to go through the origin. $\endgroup$ – gung Mar 31 '13 at 16:32
  • 6
    $\begingroup$ "physics experiment. [...] y-intercept [...] would lead to completely incorrect stuff." But if the experimental data indicates an intercept (btw, you could check whether the confidence interval for the line covers the origin), this would make me think very hard where the intercept comes from. I'm analyitical chemist. In analytical chemistry, we also have a bunch of relationships that should be linear without intercept. But they hardly ever are in practice, because of the nitty-gritty details of instruments and measurements. Thus, we usualy see suppressing the intercept as a very bad idea. $\endgroup$ – cbeleites Mar 31 '13 at 23:56
12
$\begingroup$

The Ordinary Least Squares estimate of the slope when the intercept is suppressed is:
$$ \hat{\beta}=\frac{\sum_{i=1}^N x_iy_i}{\sum_{i=1}^N x_i^2} $$

$\endgroup$
10
$\begingroup$

@gung has given the OLS estimate. That's what you were seeking.

However, when dealing with physical quantities where the line must go through the origin, it's common for the scale of the error to vary with the x-values (to have, roughly, constant relative error). In that situation, ordinary unweighted least squares would be inappropriate.

In that situation, one approach (of several possibilities) would be to take logs, subtract the x's from the y's and estimate the log-slope (of the original variables) by the mean of the differences.

Alternatively, weighted least squares could be used. In the case of constant relative error, it would reduce to using the estimator $\hat{\beta}=\frac{1}{N}\sum_{i=1}^N \frac{y_i}{x_i}$ (the average of all the slopes through the origin).

There are other approaches (GLMs for example), but if you're doing it on a calculator, I'd lean toward my first suggestion.

You should also consider the appropriateness of any assumptions you make.


I thought it might be instructive to add the derivation of the WLS line through the origin and then my "average of slopes" and gungs OLS are special cases:

The model is $y_i=\beta x_i+\varepsilon_i\,,$ where $\text{Var}(\varepsilon_i)=w_i\sigma^2$

We want to minimize $S = \sum_i w_i(y_i-\beta x_i)^2$

$\frac{\partial S}{\partial \beta} = -\sum_i 2x_i.w_i(y_i-\beta x_i)$

Setting equal to zero to obtain the LS solution $\hat{\beta}$ we obtain $\sum w_ix_iy_i = \hat{\beta} \sum w_ix_i^2$, or $\hat{\beta}=\frac{\sum w_ix_iy_i}{\sum w_ix_i^2}$.

When $w_i\propto 1$ for all $i$, this yields gung's OLS solution.

When $w_i \propto 1/x_i^2$ (which is optimum for the case where spread increases with mean), this yields the above "average of slopes" solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.