Is a spline interpolation considered to be a nonparametric model?

Question

I am aware of the basic differences between nonparametric and parametric statistics. In parametric models, we assume the data follows a distribution and fit it onto it using a fixed number of parameters. With KDE for instance, this is not the case because we don't assume that the modeled distribution has a particular shape.

I am wondering how this relates to interpolation in general, and to spline interpolation in specific. Are all interpolation approaches considered to be nonparametric, are there "mixed" approaches, what is the case with spline interpolation?

Answer 1

This is a good question. Frequently, one will see smoothing regressions (e.g., splines, but also smoothing GAMs, running lines, LOWESS, etc.) described as nonparametric regression models.

These models are nonparametric in the sense that using them does not involve reported quantities like $\widehat{\beta}$, $\widehat{\theta}$, etc. (in contrast to linear regression, GLM, etc.). Smoothing models are extremely flexible ways to represent properties of $y$ conditional on one or more $x$ variables, and do not make a priori commitments to, for example, linearity, simple integer polynomial, or similar functional forms relating $y$ to $x$.

On the other hand, these models are parametric, in the mathematical sense that they indeed involve parameters: number of splines, functional form of splines, arrangement of splines, weighting function for data fed to splines, etc. In application, however, these parameters are generally not of substantive interest: they are not the exciting bit of evidence reported by researchers… the smoothed curves (along with CIs and measures of model fit based on deviation of observed values from the curves) are the evidentiary bits. One motivation for this agnosticism about the actual parameters underlying a smoothing model is that different smoothing algorithms tend to give pretty similar results (see Buja, A., Hastie, T., & Tibshirani, R. (1989). Linear Smoothers and Additive Models. The Annals of Statistics, 17(2), 453–510 for a good comparison of several).

If I understand you, your "mixed" approaches are what are called "semi-parametric models". Cox regression is one highly-specialized example of such: the baseline hazard function relies on a nonparametric estimator, while the explanatory variables are estimated in a parametric fashion. GAMs—generalized additive models—permit us to decide which $x$ variables' effects on $y$ we will model using smoothers, which we will model using parametric specifications, and which we will model using both all in a single regression.

Answer 2

Strictly speaking, every model is parametric in the sense of having parameters. When we speak of a "nonparametric model", we really mean a model with the number of parameters being manageable.

The technical definition of "nonparametric" just says "infinite or unspecified", but in practice it means "infinite, or so large that thinking in terms of the parameters becomes unwieldy and/or not useful". You give the example of a KDE, but a KDE is calculated from the sampled values, and the number of samples is finite, so the set of samples is technically a finite set of parameters.

If each spline has a finite number of parameters, and there is a finite number of splines, then it follows that the total number of parameters is finite, but practically speaking the number may be so large that it's not treated as parametric.

On the other hand, if the number of splines is small enough, and the models within the splines are simple enough, that may still be treated as being parametric. Other factors are whether there's a large collection of models with the same type of parameters (that is, the parameters have different values, but the parameters from one model are analogous to those of another), and how intuitive the meaning of the parameters are.

For instance, if you model the volume of $H_2O$ as a function of temperature, you'll probably want separate splines for ice, water, and steam. If you model each as being linear with respect to temperature, you have one coefficient of expansion for each phase (and probably different intercepts as well), which is a small enough number of parameters to being considered "parametric". You'll then also have solid, liquid, and gas coefficients of expansion for other substances.

In this case, the small number of parameters for a particular substance, the large number of substances that have those types of parameters, and the straight forward meaning of the parameters (how much does the substance expand when you heat it) contribute to it likely being considered a parametric model.