1
$\begingroup$

Consider the model $y_i = \beta x_i + \varepsilon_i$ (simple linear regression without an intercept).

In this setting, the $i$th fitted value is

$$\hat{y}_i = \hat{\beta}x_i$$

Where

$$\hat{\beta} = \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$

I am being asked to find an expression for $\alpha_{i'}$ such that we can write

$$\hat{y}_i = \sum_{i'=1}^n \alpha_{i'} y_{i'}$$

That is, the fitted values are linear combinations of $y_{i'}$.


Aside from finding $\alpha_{i'}$, I am confused as to why the index changes to $i'$ in the denominator of $\hat{\beta}$. What's the point in doing this and what is $\alpha_{i'}$?

For reference, this is exercise 5 from chapter 3 of an Introduction to Statistical Learning.

Consider the fitted values that result from performing linear regression without an intercept. In this setting, the ith fitted value takes the form $$\hat{y}_i = x_i \hat{\beta}$$ where $$\hat{\beta} = \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$ Show that we can write $$\hat{y}_i = \sum_{i'=1}^n \alpha_{i'}y_{i'}$$ What is $\alpha_{i'}$?

Note: We interpret this result by saying that the fitted values from linear regression are linear combinations of the response values.

$\endgroup$
2
  • $\begingroup$ Does $(X^TX)^{-1} X^Ty$ look familiar to you? $\endgroup$
    – Dave
    Mar 21, 2022 at 3:15
  • 1
    $\begingroup$ There is no mathematical reason to use the symbol "$i^\prime$" in the summation. Some authors do this to avoid any possibility of confusing this "bound variable" with the symbol "$i$" employed in the sum in the numerator. $\endgroup$
    – whuber
    Mar 22, 2022 at 18:56

1 Answer 1

1
$\begingroup$

So I stared at the question long enough. Turns out it's just a matter of playing around with the summation's indices.

If we substitute $\hat{\beta}$ into $\hat{y}_i$ $$\hat{y}_i = x_i \frac{\sum_{i=1}^n x_i y_i}{\sum_{i'=1}^n x_{i'}^2}$$

For the purpose of both summations, the $x_i$ on the left is a constant. Hence, if we change the indices to match the question \begin{align} \hat{y}_i & = x_i \frac{\sum_{i'=1}^n x_{i'} y_{i'}}{\sum_{j=1}^n x_j^2} \\ & = \frac{\sum_{i'=1}^n x_ix_{i'} y_{i'}}{\sum_{j=1}^n x_j^2} \\ & = \sum_{i'=1}^n \frac{x_i x_{i'}}{\sum_{j=1}^n x_j^2} y_{i'} \\ & = \sum_{i'=1}^n \alpha_{i'} y_{i'} \end{align}

Where $$\alpha_{i'} = \frac{x_i x_{i'}}{\sum_{j=1}^n x_j^2}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.