I'm working on Chapter 5 from The elements of statistical learning which describes the more general linear models that is splines. The fragment (chapter 5, page 143) below describes the alternative and more direct method of building continuous splines. Chapter 5, page 143

To be honest I don't really know how does that base work. The previous method used indicators and then two constraints for continuity which was quite intuitive. What does ''the positive'' part mean?

Is the final function $f(X) = \sum \beta_i h_i(X)$ just a sum of $h_1, h_2, h_3, h_4$ described above or do I need some indicators too?

  • $\begingroup$ Visually, it's the first panel of the third display in my question here: stats.stackexchange.com/questions/284369/…. The "positive part" function assigns 0 to every otherwise negative value. So $X_+$ would be 0 for every $X<0$ and then $X$ for $X>0$ $\endgroup$ – AdamO Dec 19 '19 at 15:26
  • $\begingroup$ @AdamO Thank you. So know after choosing the base $h_1$, $h_2$, $h_3$, $h_4$ my model would be $f(X) = \sum \beta_i h_i(X)$? Without any indicators? $\endgroup$ – Hendrra Dec 19 '19 at 15:29
  • 1
    $\begingroup$ Correct. The $_+$ function is a convenience function. Formally it could be written $x_+ = x \mathcal{I} (x>0)$ so you see how it informally uses indicators without you having to write a lot of notation. $\endgroup$ – AdamO Dec 19 '19 at 15:35
  • $\begingroup$ @AdamO Thank you! I was pretty surprised that now it is needed only two indicators. $\endgroup$ – Hendrra Dec 19 '19 at 15:36

The positive part of a function $f(x)$, noted $f^+(x)$ or $f(x)_+$, is defined as: $$ f(x)_+ = \max(f(x),0) $$

For example the function $h_3(x) = (X-\xi_1)_+ = \max(X-\xi_1,0)$. That is, whenever $X \leq \xi_1$, $X-\xi_1 \leq 0$ and $\max(X-\xi_1,0)=0$. And for $X \geq \xi_1$, $(X-\xi_1)_+= X-\xi_1$.

So if $h_1,h_2,\dots$ are defined using positive parts, then $\sum \beta_ih_i(x)$ already include some indicators (inside positive parts).

Just for information, the negative part of a function is defined as

$$ f(x)_- = - \min(f(x),0) $$

And we have $f(x) = f(x)_+ - f(x)_-$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.