11
$\begingroup$

Linear splines are easy to discuss. Knots are where the slopes change, and only one level of continuity is enforced.

When discussing cubic splines (with the usual 3 levels of continuity) or natural cubic splines (linear tail restricted cubic splines) I often speak loosely as "a knot is where a curvature change happens" or where a "shape change happens". For functions, the formal definition of curvature has the second derivative dominating the calculation, and so formally speaking the curvature is changing everywhere in a cubic spline function.

What is the best language to use? Should we say "the knots are where we allow shape changes"? Where we allow rapid shape changes? Or is it best to be this explicit: the knots are points where there are changes in jolt (jerk; 3rd derivative) of the function? Other ideas?

Based on comments below, the best language I can think of at the moment is the following, for a general audience:

Knots are where different cubic polynomials are joined, and cubic splines force there to be three levels of continuity (the function, its slope, and its acceleration or second derivative (slope of the slope) do not change) at these points. At the knots the jolt (third derivative or rate of change of acceleration) is allowed to change suddenly, meaning the jolt is allowed to be discontinuous at the knots. Between knots, jolt is constant.

Alternative version:

Knots are where cubic polynomials are joined, and continuity restrictions make the joins invisible. The function, its slope, and its acceleration (slope of slope; second derivative) do not change at a knot. But the rate of change of the acceleration (jolt; third derivative) is allowed to change abruptly at a knot.


It would still be nice if there is a simpler term not unlike "shape change" that could be used to describe this in addition to the technically correct wording.


Update: I added a graph showing a cubic spline function and its first three derivatives in Section 2.4.4 of my Regression Modeling Strategies course notes.

$\endgroup$
3
  • 3
    $\begingroup$ You are right about the curvature (even though many references conflate the second derivative with the curvature). At stats.stackexchange.com/a/43075/919 I was discussing the same phenomenon (there loosely described as "kinks" or "cusps"). In looking back at my answer, I see that for clarity I elected to characterize these points as places where derivatives through a particular order are discontinuous. I hesitate, though, to recommend any language without understanding the intended audience. $\endgroup$
    – whuber
    Aug 13, 2019 at 14:27
  • 2
    $\begingroup$ Yes by change in 3rd derivative I could add that the 3rd derivative of a cubic spline is actually discontinuous at the knot, so as you said it is a point of discontinuity of the 3rd derivative. But for general use we may find better language. $\endgroup$ Aug 13, 2019 at 15:27
  • 4
    $\begingroup$ Re the edit: "shape changes" doesn't seem right, because "shape" has a fairly well defined meaning in math and stats, as well as colloquially, which differs from the looser one you are invoking here. ("Shape" is whatever properties persist under the action of a geometric group such as all rotations, translations, and isotheties, or under all recentering and rescaling of $x$ and rescaling of $y$ in a plot, etc.). Somehow you have to convey the idea of a continuous 2nd derivative, something like "if you drove on this path at constant speed, you would feel no sudden change in acceleration." $\endgroup$
    – whuber
    Aug 13, 2019 at 20:12

2 Answers 2

7
$\begingroup$

This answer on math.stackexchange.com suggests one way to proceed. In particular:

The typical mathematical definition of "smooth" says something about how many continuous derivatives the function has. But these sorts of definitions bear little relationship to the intuitive notion of "smoothness" of a curve.

Starting from the limitations of fitting an infinitely mathematically smooth (in the sense of infinite differentiability) high-degree polynomial might help heuristically. I'd suggest something like:

At knots the required level of mathematical smoothness is relaxed, better to match the intuitive notion of smoothness while allowing the curve to pass through the knots. The curve between each pair of adjacent knots can then be a simple, infinitely smooth, 3rd-degree polynomial.

If there is time, an illustration like the following based on Runge's phenomenon might help.

Consider the following 9 points joined with straight lines:

Piecewise Linear Interpolation

We want to fit a smooth curve to these points, to avoid the sharp changes in the line at the points. We could try to fit a curve that is infinitely smooth mathematically through these points, in the sense that not only is the curve continuous, but the slope of the curve is continuous, as is the slope of the slope, and so on forever (infinite differentiability). Polynomials are infinitely smooth in that sense, but here's what you get if you fit a polynomial through these points:

Interpolating Polynomial

As @bubba has put it about the high-degree polynomials needed for this type of fitting:

No-one (except a mathematician) would call them "smooth".

If we remove the requirement for infinite mathematical smoothness at the knots, however, we can do much better. Then we can use an infinitely smooth 3rd-degree polynomial between each pair of adjacent knots, and at the knots require just the minimum smoothness needed to make the joins invisible:

Cubic spline with Runge function

where the orange line is a cubic spline fit and the blue line shows the smooth Runge function from which the points were sampled. This approach provides "the least possible amount of wiggling in between" the knots and thus meets an intuitive sense of "smoothness."

$\endgroup$
2
  • 3
    $\begingroup$ This will be extremely useful in my teaching @EdM. Thank you. It doesn't solve directly the "how to say what happens at knots in a brief phrase" question, but is very helpful nonetheless. $\endgroup$ Aug 18, 2019 at 11:11
  • 3
    $\begingroup$ @FrankHarrell to this non-mathematician's ears, something like "a change from $C^{\infty}$ to $C^2$ smoothness at the knots" (maybe more precise: in neighborhoods containing a knot) sounds OK and agrees with terms I recall from high school calculus. A "discontinuity in the jerk" would be equivalent and correct, but this technical use of "jerk" or "jolt" doesn't seem to be very common outside of kinematics, and to my cynical ears the phrase as a whole sounds too much like a change of Dean at an academic institution. $\endgroup$
    – EdM
    Aug 18, 2019 at 16:53
1
$\begingroup$

Disclaimer: I'm not a native speaker and I learned about cubic splines many years ago. Taking this into account:

Both of your proposals, staring with

Knots are where different cubic polynomials are joined

are clear to me, although I didn't know the technical meaning of "jolt". But, the way you state it has a touch -- at least to my ear -- of implying that knots are some properties of the curve; that we first construct the curve and then find the knots on it based on the properties you list. It is like saying

"An extreme is where the first derivative is zero".

There is a causal implication here: First we have a curve (which might follow e.g. from the laws of physics) and on that curve we find the extremes.

Since for splines it's the other way round -- we first choose the knots, and then construct the curve around them, -- I'd start from the cubic segments and explain that points where they are joined are called "knots". Something like:

(These) different cubic polynomial segments are joined together to produce a (visually) smooth curve. In order to achieve this, cubic splines enforce three levels of continuity: the function, its slope, and its acceleration or second derivative (slope of the slope) do not change at the joints. Only the jolt -- the third derivative of the curve -- may abruptly change. These joints are called "knots".

$\endgroup$
1
  • $\begingroup$ I would say that the curves are properties of the knots. $\endgroup$ Mar 9 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.