9
$\begingroup$

I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. enter image description here

I am not clear on what the equation on the Y-axis means precisely.

My understanding is that $\beta$ is a vector of length 31 with first 10 elements iid as $N(0,0.4)$ and the other 21 elements being 0.

Then $\hat{\beta}(k)$ represents the coefficient estimates at step k in the stepwise regression. For example, in forward stepwise, after the first coefficient is added we would have $\hat{\beta}(k=1)$ which would be a vector of length 31 with the first element being the first coefficient estimate and the remaining 30 elements being 0. Then, sequentially, the remaining 30 coefficients would be estimated, updating this vector for each new coefficient.

At each step $k$ we would calculate $\mathrm{E}||\hat{\beta}(k) - \beta||^2$ by calculating $||\hat{\beta}(k) - \beta||^2$ and averaging across the 50 repetitions.

What is the correct interpretation for this equation in this context?

$\endgroup$
2
  • 5
    $\begingroup$ I agree with your understanding of the problem, but then I am not sure what interpretation you are expected for the equation. It represents the mean squared error of the estimated coefficients of the k-th model compared to the true coefficients, i.e. how close the k-th model is from the true model in terms of coefficients estimates (not predictions). $\endgroup$
    – Guillem
    Commented Jul 1, 2019 at 9:08
  • $\begingroup$ @Guillem: your comment looks like an answer to me. Do you want to post it as such? Better to have a short answer than no answer at all. Anyone who has a better answer can post it. Incidentally, the "E" in the formula is a little misleading, since what is shown is not an expectation, but an average over multiple runs, which is an estimation of the expectation. $\endgroup$ Commented Nov 4, 2021 at 7:35

1 Answer 1

1
$\begingroup$

This answer is adapted from my comment above.

Your understanding of the problem is correct in my opinion. You can interpret the equation as the mean squared error of the estimated coefficients of the $k$-th model compared to the true coefficients, i.e. how close the $k$-th model is from the true model in terms of coefficients estimates. As opposed to evaluating predictions, this metric will tell you how well you modelled the data-generating process.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.