# If a regression is fit without an intercept, why are the fitted values a linear function of the response?

Consider the model $$y_i = \beta x_i + \varepsilon_i$$ (simple linear regression without an intercept).

In this setting, the $$i$$th fitted value is

$$\hat{y}_i = \hat{\beta}x_i$$

Where

$$\hat{\beta} = \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$

I am being asked to find an expression for $$\alpha_{i'}$$ such that we can write

$$\hat{y}_i = \sum_{i'=1}^n \alpha_{i'} y_{i'}$$

That is, the fitted values are linear combinations of $$y_{i'}$$.

Aside from finding $$\alpha_{i'}$$, I am confused as to why the index changes to $$i'$$ in the denominator of $$\hat{\beta}$$. What's the point in doing this and what is $$\alpha_{i'}$$?

For reference, this is exercise 5 from chapter 3 of an Introduction to Statistical Learning.

Consider the fitted values that result from performing linear regression without an intercept. In this setting, the ith fitted value takes the form $$\hat{y}_i = x_i \hat{\beta}$$ where $$\hat{\beta} = \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$ Show that we can write $$\hat{y}_i = \sum_{i'=1}^n \alpha_{i'}y_{i'}$$ What is $$\alpha_{i'}$$?

Note: We interpret this result by saying that the fitted values from linear regression are linear combinations of the response values.

• Does $(X^TX)^{-1} X^Ty$ look familiar to you?
– Dave
Commented Mar 21, 2022 at 3:15
• There is no mathematical reason to use the symbol "$i^\prime$" in the summation. Some authors do this to avoid any possibility of confusing this "bound variable" with the symbol "$i$" employed in the sum in the numerator.
– whuber
Commented Mar 22, 2022 at 18:56

If we substitute $$\hat{\beta}$$ into $$\hat{y}_i$$ $$\hat{y}_i = x_i \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$
For the purpose of both summations, the $$x_i$$ on the left is a constant. Hence, if we change the indices to match the question \begin{align} \hat{y}_i & = x_i \frac{\sum_{i'=1}^n x_{i'} y_{i'}}{\sum_{j=1}^n x_j^2} \\ & = \frac{\sum_{i'=1}^n x_ix_{i'} y_{i'}}{\sum_{j=1}^n x_j^2} \\ & = \sum_{i'=1}^n \frac{x_i x_{i'}}{\sum_{j=1}^n x_j^2} y_{i'} \\ & = \sum_{i'=1}^n \alpha_{i'} y_{i'} \end{align}
Where $$\alpha_{i'} = \frac{x_i x_{i'}}{\sum_{j=1}^n x_j^2}$$