3
$\begingroup$

Im looking through Section 5.2.1 of Elements of statistical learning, and am confused by this section

enter image description here

enter image description here

I understand that, from this answer , the definition of $d_k(X)$ is important in order to derive the following:

enter image description here enter image description here

My questions are:

  1. What does this $d_k(X)$ term represent? I can't figure out what is is.

  2. Why is $N_{k+2}$ then given as it is above, using these $d(X)$ terms I am uncertain about?

  3. Why are there only 3 basis functions shown here, when above it says there are K basis functions. Does this mean there will be a $N_{3}, N_{4},...$? If so, would they take the same format as $N_{k+2}$?

  4. If we are representing a natural cubic spline, then where are the $X^2$ and $X^3$ basis functions?

$\endgroup$

1 Answer 1

5
$\begingroup$

0. Your question suggests (by use of the word "the") that there is only one basis for the natural cubic splines. There are in fact many, and this is only one particular basis for them.

1. It sounds like you want an intuitive way to interpret $d_k(x)$. Nice, intuitive ways of understanding things don't always exist, and I doubt such an interpretation exists for the $d_k$. Hastie, Tibshirani, and Friedman certainly don't mention one. Nonetheless, we can visualize the functions $d_k(x)$, however, which might help. Let's take the knots to be the set $\{ 0, 1/3, 2/3, 1 \}$. Then we can plot the functions $d_k$ and $N_k$, along with an example of a natural cubic spline given by the linear combination of the $N_k$ with weights $(\beta_1, \beta_2, \beta_3, \beta_4) = (1, 10, -50, 100)$.

enter image description here

enter image description here

enter image description here

4. (I know this is out of order) The natural cubic splines differ from cubic splines in that they are linear outside the interval $(\xi_1, \xi_K)$. The functions $x^2$ and $x^3$ can't be part of the basis because they are not linear outside the interval $(\xi_1, \xi_K)$. Any function in the basis has to itself be an element of the function space, and $x^2$ and $x^3$ are not natural cubic splines because they don't satisfy the linearity condition outside $(\xi_1, \xi_K)$.

2. Now, the reason we need to the basis functions $N_k$ is the same reason that $x^2$ and $x^3$ can't be basis functions: every basis function must be linear outside the interval $(\xi_1, \xi_K)$. The $N_k$ are constructed in the post you linked in order to satisfy this constraint. By taking the differences $d_k(x) - d_{K - 1}$, the quadratic and cubic parts of these functions cancel out on the interval $(\xi_K, \infty)$, which makes the resulting functions linear on that interval.

3. Perhaps the authors could have mentioned that $k$ is an index/variable that ranges over the values $k = 1, \dots, K-2$. This means that the expression $N_{k + 2} (x) = d_k (x) - d_{K - 1} (x)$ for $k = 1, \dots, K-2$ represents the $3^{rd}, 4^{th}, \dots, K^{th}$ basis functions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.