# Regression without intercept: deriving $\hat{\beta}_1$ in least squares (no matrices)

In An Introduction to Statistical Learning (James et al.), in section 3.7 exercise 5, it states that the formula for $\hat{\beta}_1$ assuming linear regression without an intercept is $$\hat{\beta}_1 = \dfrac{\displaystyle\sum\limits_{i=1}^{n}x_iy_i}\sum\limits_{i=1}^{n}x_i^2}\text{,$$ where $\hat{\beta}_0 = \bar{y}-\hat{\beta}_1\bar{x}$ and $\hat{\beta}_1 = \dfracS_{xy}}{S_{xx}$ are the usual estimates under OLS for simple linear regression ($S_{xy} = \displaystyle\sum\limits_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})$).

This is not the actual exercise; I am merely wondering how to derive the equation. Without using matrix algebra, how do I derive it?

My attempt: with $\hat{\beta}_0 = 0$, we have $\hat{\beta}_1 = \dfrac{\bar{y}}{\bar{x}} = \dfrac{S_{xy}}{S_{xx}}$.

After some algebra, it can be shown that $\displaystyle S_{xy} = \sum\limits_{i=1}^{n}x_i^2 - n\bar{x}\bar{y}$ and $\displaystyle S_{xx} = \sum\limits_{i=1}^{n}x_i^2 - n\bar{x}^2$. From here, I'm stuck.

• The formula is immediate from the geometric interpretation of least squares, using $x/||x||$ as a "matcher" for $y$ and recognizing the formula for $(\hat\beta_1)x$ as being $(y\cdot (x/||x||))x/||x||$.
– whuber
Jul 2 '15 at 21:02
• @whuber : Rather than writing $x/||x||,$ I would write $x/\|x\|.$ If that's not conspicuous enough for you, consider the typographical difference between $||x|| ||y||,$ coded as ||x|| ||y||, and $\|x\|\|y\|,$ coded as \|x\|\|y\|. $\qquad$ Feb 3 '19 at 18:31

This is straightforward from the Ordinary Least Squares definition. If there is no intercept, one is minimizing $R(\beta) = \sum_{i=1}^{i=n} (y_i- \beta x_i)^2$. This is smooth as a function of $\beta$, so all minima (or maxima) occur when the derivative is zero. Differentiating with respect to $\beta$ we get $-\sum_{i=1}^{i=n} 2(y_i- \beta x_i)x_i$. Solving for $\beta$ gives the formula.