Proof that quadratic regression splines are continuous at the knots?

Question

The title says it all. For more info: I have a dependent and independent variable $y$ and $X$. I want to fit a square spline on the data given a single knot $k$. I can do that by fitting 2 separate square functions in the areas $X\le k$ and $X>k$ with the constraint that they have to have a common point at $X=k$ by fitting (OLS) the following regression equation:

$$ y_t = \beta_0 + \beta_1 X_t + \beta_2 X_t^2 + \beta_3 (X_t-k)_+ + \beta_4(X_t-k)_+^2 + \varepsilon_t $$

where $(X-k)_+ = max(X-k, 0)$ is a truncated function.

However, the resulting fit is discontinuous at $k$, e.g: (here $k=2.5$)

To make the fit continuous at $k$ one just needs to drop the lower-order terms of the truncated function, i.e. this is the same as above without $\beta_3 (X_t-k)_+ $ (see also this video): $$ y_t = \beta_0 + \beta_1 X_t + \beta_2 X_t^2 + \beta_4(X_t-k)_+^2 + \varepsilon_t $$

Now I get a continuous function (in the first derivative) at $X=k$:

The question is: Why?

If I take the derivative of the functions before and after $k$ I get: $$f'(X)=\beta_1 + 2\beta_2 X_t + 2\beta_4 (X_t>k) $$

• before $k$: $f'(k)=\beta_1 + 2\beta_2 k + 0 $
• after $k$ (meaning something like $\lim_{X\to k, X > k}{(f'(X))}$: $f'(k)=\beta_1 + 2\beta_2 k + 2\beta_4k $, since $ (X_t-k)_+$ is only active if $X_t>k$.

So the two are different, one includes $2\beta_4k$ the other one doesn't. How are they the same?

Answer 1

The prediction formula is:

$$ E(y) = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_4(X-k)_+^2 $$

Writing out the function and its derivative, both for $X < k$ and $X \geq k$, we can check whether the function and its derivative are continuous at $X = k$ (i.e., return the same value at $x = K$):

Before $k$

If $X \leq k$, the prediction function is given by:

$$ E(y|X < k) = \beta_0 + \beta_1 X + \beta_2 X^2 $$

The first derivative w.r.t. $X$ is:

$$ \beta_1 + 2 \beta_2 X $$

At $X = k$, this first derivative equals:

$$ \beta_1 + 2 \beta_2 k $$

After $k$

If $X \geq k$, the function is:

$$ E(y|X \geq k) = \beta_0 + \beta_1 X + \beta_2 X^2 + \beta_4(X-k)^2 $$

The first derivative w.r.t. $X$ is:

$$ \beta_1 + 2 \beta_2 X + 2\beta_4(X-k) $$

At $X = k$, this first derivative equals:

$$ \beta_1 + 2 \beta_2 k $$

Thus, the first derivative is continuous at $X=k$, because both first derivatives are identical at $X = k$.

Answer 2

Overall continuity of the regression spline comes from the fact that it is a polynomial of constituent parts that are all continuous functions. To demonstrate this we need to show that the shifted ramp term $(x-k)_+$ is continuous. To facilitate this analysis, for any $k \in \mathbb{R}$ we will use the shifted ramp function $f_k: \mathbb{R} \rightarrow \mathbb{R}$ defined by:

$$f_k(x) \equiv (x-k)_+ = \begin{cases} 0 & & \text{for } x < k \\[6pt] x-k & & \text{for } x \geqslant k \\[6pt] \end{cases}$$

For any values $a,x \in \mathbb{R}$ we obtain:

$$f_k(x) - f_k(a) = (x-k)_+ - (a-k)_+ = \begin{cases} 0 & & \text{for } x < k \text{ and } a < k \\[6pt] x-k & & \text{for } x \geqslant k \text{ and } a < k \\[6pt] k-a & & \text{for } x < k \text{ and } a \geqslant k \\[6pt] x-a & & \text{for } x \geqslant k \text{ and } a \geqslant k \\[6pt] \end{cases}$$

which implies that:$^\dagger$

$$\min(0,x-a) \leqslant f_k(x) - f_k(a) \leqslant \max(0,x-a).$$

We therefore have:

$$0 \leqslant |f_k(x) - f_k(a)| \leqslant |x-a|.$$

Consequently, for any value $\varepsilon > 0$ we can take $\delta(\varepsilon) = \varepsilon$ and we have:

$$0 \leqslant |f_k(x)-f_k(a)| \leqslant \varepsilon \quad \quad \quad \text{for all } 0 \leqslant |x-a| \leqslant \delta(\varepsilon).$$

This establishes that $\lim_{x \rightarrow a} f_k(x) = f_k(a)$ which means that $f_k$ is a continuous function. The remainder of what you want to demonstrate follows from the fact that polynomials of continuous functions are also continuous.

---

$^\dagger$ To obtain these bounds, just maximise/minimise $f_k(x) - f_k(a)$ with respect to $k$ for each case.

Answer 3

If I take the derivative of the functions before and after $k$ I get: $$f'(X)=\beta_1 + 2\beta_2 X_t + 2\beta_4 (X_t>k) $$

• before $k$: $f'(k)=\beta_1 + 2\beta_2 k + 0 $
• after $k$ (meaning something like $\lim_{X\to k, X > k}{(f'(X))}$: $f'(k)=\beta_1 + 2\beta_2 k + 2\beta_4k $, since $ (X_t-k)_+$ is only active if $X_t>k$.

What you did by taking the derivatives is not very clear. Especially the meaning of $(X_t>k)$ is not very clear.

I get something different instead

$$f'(X)=\beta_1 + 2\beta_2 X_t + 2\beta_4 (X_t-k)_+$$

• before $k$: $f'(k)=\beta_1 + 2\beta_2 X_t + 0 $

• after $k$: $f'(k)=\beta_1 + 2\beta_2 X_t + 2\beta_4(X_t-k) $