1
$\begingroup$

I am reading ISL and found that fitting a cubic spline with $K$ knots uses $K+4$ degrees of freedom, since it estimates $K+4$ regression coefficients (p.273). However, in ESL they say that a natural cubic spline with $K$ knots is represented by $K$ basis functions (p.145), however according to ISL (p.275) these are $K+1$ degrees of freedom. My understanding is that since it requires fitting $K$ basis functions (the first of which is the intercept) it should also use $K$ degrees of freedom. What am I missing here?

$\endgroup$
2
  • $\begingroup$ Did you read the footnote on page 275 of ISLR? $\endgroup$
    – angryavian
    Oct 3 '20 at 23:46
  • $\begingroup$ I did yes, and it does make sense. But if you think of the $K$ final basis functions that you fit (with no additional constraints), I do not see where that extra parameter comes from. $\endgroup$
    – econ86
    Oct 4 '20 at 0:01
1
$\begingroup$

$K$ is then number of knots, including the exterior knots (beyond which the natural splines are constrained to be linear). The example on pages 274-5 of ISLR puts three knots on the 25th, 50th, and 75th percentiles, but also two more knots on the boundaries of the data. [This was not explicitly explained in the body of the text, but it was mentioned in the footnote; you can also infer this based on the fact that the left panel of Figure 7.5 is not linear below the 25th percentile or above the 75th percentile.] So in that example, $K=5$, not $3$. The footnote then goes on to say that this "results in $9-4=5$ degrees of freedom" which equals $K$ and does not contradict ESL. (I am not sure why the last sentence of the footnote concludes with $4$; for some reason they removed the degree of freedom corresponding to the intercept, which contradicts the convention they used when counting $K+4$ degrees of freedom in the previous section on cubic splines.)

Link to ISLR text


Response to comment:

$K$ counts all knots, including the two boundary knots. If you insist on counting interior knots, perhaps it would help make the distinction clearer by introducing the notation $K_{\text{interior}} = K-2$. The claim then is a natural spline with $K$ knots (or $K_{\text{interior}} = K-2$ interior knots) has $K = K_{\text{interior}}+2$ degrees of freedom. This is what ESL claims; the example in ISL has $K=5$ and the footnote states "this results in $5$ degrees of freedom." There is no "$K+1$" in ISL; if anything, that example is somehow resulting in $K-1$ because they inexplicably throw away the intercept degree of freedom.

$\endgroup$
2
  • $\begingroup$ Thanks. Part of my confusion came from the change in convention you mentioned. But also one point remains: if $K$ interior knots require $K+1$ degrees of freedom but only fit $K$ parameters, where is that extra parameter coming from? $\endgroup$
    – econ86
    Oct 4 '20 at 8:44
  • $\begingroup$ @econ86 see my update $\endgroup$
    – angryavian
    Oct 4 '20 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.