45
votes
Accepted
Why is the use of high order polynomials for regression discouraged?
I cover this in some detail in Chapter 2 of RMS. Briefly, besides extrapolation problems, ordinary polynomials have these problems:
The shape of the fit in one region of the data is influenced by ...
- 86.9k
44
votes
Accepted
Adaptive GAM smooths in mgcv
Most of the extra smooths in the mgcv toolbox are really there for specialist applications — you can largely ignore them for general GAMs, especially univariate smooths (you don't need a random ...
- 45.8k
36
votes
Why is the use of high order polynomials for regression discouraged?
Yes, polynomials are also problematic in interpolation, because of overfitting and high variability.
Here is an example. Assume your dependent variable $y$ is uniformly distributed on the interval $[0,...
- 118k
27
votes
How different are restricted cubic splines and penalized splines?
From my reading, the two concepts you ask us to compare are quite different beasts and would require an apples and oranges-like comparison. This makes many of your questions somewhat moot — ideally (...
- 45.8k
24
votes
Accepted
Splines vs Gaussian Process Regression
I agree with @j__'s answer.
However, I would like to highlight the fact that splines are just a special case of Gaussian Process regression/kriging.
If you take a certain type of kernel in Gaussian ...
- 1,576
24
votes
Accepted
The definition of natural cubic splines for regression
Let's start by considering ordinary cubic splines. They're cubic between every pair of knots and cubic outside the boundary knots. We start with 4df for the first cubic (left of the first boundary ...
- 278k
23
votes
Selecting knots for a GAM
Where is the idea coming from that GCV will automatically choose the number of knots? The number of knots (i.e., the basis dimension) is fixed and cannot be changed during model fit. What the GCV ...
- 652
23
votes
Why should binning be avoided at all costs?
It is a slight exaggeration to say that binning should be avoided at all costs, but it is certainly the case that binning introduces bin choices that introduce some arbitrariness to the analysis. ...
- 118k
22
votes
Accepted
Splines in GLM and GAM
You are mistaken. Splines have a linear representation using derived covariates. As an example, a quadratic trend is non-linear, but can be modeled in a linear model by taking: $E[Y|X] = \beta_0 + \...
- 60.6k
22
votes
Accepted
Is a spline interpolation considered to be a nonparametric model?
This is a good question. Frequently, one will see smoothing regressions (e.g., splines, but also smoothing GAMs, running lines, LOWESS, etc.) described as nonparametric regression models.
These models ...
- 29.3k
21
votes
Smoothing methods for gam in mgcv package?
mgcv uses a thin plate spline basis as the default basis for it's smooth terms. To be honest it likely makes little difference in many applications which of these you choose, though in some situations ...
- 45.8k
21
votes
Accepted
Use of splines in parameter estimation
I found their code on the Wayback machine and they used the smooth.spline-function in R. The paper points to http://genomine.org/qvalue/results.html for code and ...
- 2,197
20
votes
Accepted
B-Splines VS high order polynomials in regression
I would usually only consider splines rather than polynomials. Polynomials cannot model thresholds and are often undesirably global, i.e., observations at one range of the predictor have a strong ...
- 118k
20
votes
Accepted
Splines: relationship of knots, degree and degrees of freedom
In essence, splines are piecewise polynomials, joined at points called knots. The degree specifies the degree of the polynomials. A polynomial of degree 1 is just a line, so these would be linear ...
- 29.5k
19
votes
Accepted
GAM vs LOESS vs splines
What matters the most is the number of effective degrees of freedom that you give to each approach. For nonparametric smoothers such as loess this is controlled by the bandwidth whereas for ...
- 86.9k
19
votes
Do fractional polynomials have any advantages over restricted cubic splines?
Fractional polynomials and cubic splines each have the defects of their virtues.
The point of cubic splines is to be local and flexible and smooth and able to approximate any smooth curve.
The point ...
- 34.7k
18
votes
Accepted
Can splines be used for prediction?
From my interpretation of the question, the underlying question you are asking is whether or not you can model time as a spline.
The first question I will attempt to answer is whether or not you ...
- 1,063
16
votes
Accepted
Why are the basis functions for natural cubic splines expressed as they are? (ESL)
First it is not the basis but a basis: We want to build a basis for $K$ knots of natural cubic splines.
According to the constraints, "a natural cubic splines with $K$ knots is represented by $K$ ...
- 1,200
15
votes
Accepted
Do fractional polynomials have any advantages over restricted cubic splines?
I can think of some, not-too-compelling circumstances where fractional polynomials (FPs) would be preferable to restricted cubic splines (RCSs):
Direct interpretation of the functional form is more ...
- 42.6k
14
votes
Why is the use of high order polynomials for regression discouraged?
Runge's phenomenon can lead to high-degree polynomials being much wigglier than the variation actually suggested by the data. An appeal of splines as a substitute for high-degree polynomials, ...
- 19.8k
14
votes
Ideal Use Cases for Splines
You have to define what you mean by "ideal" or "best" in this question, but I will give you my two cents none the less.
Are there any ideal use cases for splines?
My (very ...
- 33.5k
13
votes
How should I check the assumption of linearity to the logit for the continuous independent variables in logistic regression analysis?
Logistic regression does NOT assume a linear relationship between the dependent and independent variables. It does assume a linear relationship between the log odds of the dependent variable and the ...
- 149
13
votes
Splines in GLM and GAM
@AdamO's answer is correct, in that spline-based fits can certainly be done in the standard GLM framework. That's not to say that GAM's are just a special case of GLM's though! While there are a ...
- 20.2k
13
votes
Why is the use of high order polynomials for regression discouraged?
If your goal is interpolation, you typically want the simplest function that describes your observations and avoid overfitting.
Given that it is unusual to see physical laws and relationships which ...
- 1,066
12
votes
Accepted
Motivating use of Bayesian splines in excess mortality estimation
The death rate can't be negative (the pandemic was bad but it wasn't zombie apocalypse bad), so a natural way to enforce that is to fit an additive/linear model on the log scale (hence why the model ...
- 45.8k
11
votes
Accepted
Difference between smoothing splines and splines in R
Smoothing splines have all the knots (knots at each point), but then regularizes (shrinks the coefficients/smooths the fit) by adding a roughness penalty term (integrated squared second derivative ...
- 278k
11
votes
what is the advantage of b-splines over other splines?
Splines are a large class of methods.
The method of B-splines is a simple method for taking a single covariate and expanding it such that it spans the set of all functions that are a polynomial of ...
- 20.2k
11
votes
Monotonic splines in Python
Hi I do not know the specifics of your problem but you might find the following reference really interesting: Eilers, 2006 (especially paragraph 3). The idea presented in the reference is rather ...
- 1,151
11
votes
Use of splines in parameter estimation
Edit: In light of Lukas Lohse's answer (which I think should be the accepted one!), my original answer below is misleading.
Personally I learned about splines from Tibshirani's books, where he ...
- 3,462
10
votes
Visualizing a spline basis
Here's an autoplot method for the "basis" class (which both bs and ns inherit from):
...
- 373
Only top scored, non community-wiki answers of a minimum length are eligible