I am trying to recreate FIGURE 3.6 from Elements of Statistical Learning. The only information about the figure is included in the caption. enter image description here

I am not clear on what the equation on the Y-axis means precisely.

My understanding is that $\beta$ is a vector of length 31 with first 10 elements iid as $N(0,0.4)$ and the other 21 elements being 0.

Then $\hat{\beta}(k)$ represents the coefficient estimates at step k in the stepwise regression. For example, in forward stepwise, after the first coefficient is added we would have $\hat{\beta}(k=1)$ which would be a vector of length 31 with the first element being the first coefficient estimate and the remaining 30 elements being 0. Then, sequentially, the remaining 30 coefficients would be estimated, updating this vector for each new coefficient.

At each step $k$ we would calculate $\mathrm{E}||\hat{\beta}(k) - \beta||^2$ by calculating $||\hat{\beta}(k) - \beta||^2$ and averaging across the 50 repetitions.

What is the correct interpretation for this equation in this context?

  • 3
    $\begingroup$ I agree with your understanding of the problem, but then I am not sure what interpretation you are expected for the equation. It represents the mean squared error of the estimated coefficients of the k-th model compared to the true coefficients, i.e. how close the k-th model is from the true model in terms of coefficients estimates (not predictions). $\endgroup$ – Guillem Jul 1 '19 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.