Intuition for “weights” in simple linear regression

Suppose we have data $\{x_i,y_i\}_{i=1}^n$ where $x_i \in \mathbb{R}$ and $y_i \in \mathbb{R}$ and we model $$y_i=\beta x_i + \varepsilon_i$$ The ordinary least squares estimate of $\beta$ is $$\widehat \beta = \sum_{i=1}^n w_i y_i$$ where $w_i={x_i}/{\sum_{j=1}^nx_j^2}$ can be viewed as "weights" on each $y_i$.

I've been thinking about what these "weights" mean and why they make sense but it seems to put more weight on larger values of $x_i$, which I don't quite understand why this makes sense.

Can someone help me with the intuition for why the "weights" on $y_i$ make sense? Thanks.

Note: I'm not interested in the derivation of $\widehat \beta$ or that these weights happen to minimize the sum of least squares. I'm interested in the intuition i.e. how would you explain this to the layman without math.

migrated from math.stackexchange.comApr 15 '15 at 8:10

This question came from our site for people studying math at any level and professionals in related fields.

• If we have $w_i=\frac{x_i}{\sum x_j^2}$ then $w_i$ is only increasing in $x_i$ holding the denominator fixed. If not, then it is not necessarily the case that $w_i$ is increasing in $x_i$. – Greg Apr 14 '15 at 5:18

The traditional problem is to minimize $$SSQ=\sum_{i=1}^n \Big(y_i^{(calc)}-y_i^{(exp)}\Big)^2$$ and, as you know, this gives a large influence to the largest values of the $y_i^{(exp)}$. This corresponds to the sum of squares of the absolute errors of the $y$'s $(w_i=1)$.
If instead you consider $$SSQ=\sum_{i=1}^n \Big(\frac{y_i^{(calc)}-y_i^{(exp)}}{y_i^{(exp)}}\Big)^2$$ This corresponds to the sum of squares of the relative errors of the $y$'s $(w_i=\frac 1 {y_i^2})$.
But there is another situation where the weights can be important. Suppose that the model is $y=Ae^{Bx}$ which is nonlinear. You can linearize it taking logarithms $\log(y)=\alpha+\beta x$ but ordinary least squares can lead to very different results compared to nonlinear regression since the transform gives greater weights to small $y$ values. For this very specific case, it is been found that $$SSQ=\sum_{i=1}^n y_i\Big(\alpha+\beta x_i-\log(y_i)\Big)^2$$ is very acceptable.