5
$\begingroup$

There are a number of useful answers about how to use the fitted model for a smoothing spline to make predictions given a data frame of predictor variables, such as to this question.

But, I'm struggling to comprehend what the fitted model is. Recognizing this is somewhat poorly-structured, what is the fitted model for a smoothing spline?

To add some detail:

  • Since a smoothing spline is fit by placing a knot at every value of the predictor (and then regularizing the coefficients based on a tuning parameter), is the curve between each consecutive pair of knots associated with coefficients?
  • For example, for a cubic basis function, does the curve between knots have four coefficients (for the intercept and linear, quadratic, and cubic functions of the predictor)?
  • Since they are regularized, are only some regions of the predictor variable associated with coefficients?
$\endgroup$

1 Answer 1

4
$\begingroup$

(Most of these comments apply to spline fits more generally, including regression splines and penalized splines, not just smoothing splines, though I'll make at least some reference to smoothing splines.)

With splines the fitted model is usually written in terms of basis functions for which you estimate coefficients $\hat{y}(x)=\sum_j \hat{\phi}_j(x)$. You can write it in a number of other ways. One of those ways is as the piecewise polynomials - cubics for a cubic spline - that we're notionally trying to estimate. For another example, a fitted spline can also be written as a linear function of the data -- $\hat{y} = Ay$. Each of these representations is convenient in some situations.

In the case of smoothing splines it's reasonably easy to calculate $A$ fairly directly (or better, to compute something else from which the A could be obtained - direct computation of A isn't always numerically stable), and so that might be the form in which the smoothing spline is available. If the $x$'s are equispaced this is particularly simple.

is the curve between each consecutive pair of knots associated with coefficients?

Yes. It's directly related to the coefficients of the basis functions, but you can write the coefficients of the local (intra-knot segment) cubic in terms of those.

For example, for a cubic basis function, does the curve between knots have four coefficients (for the intercept and linear, quadratic, and cubic functions of the predictor)?

Yes, but you don't estimate those coefficients directly, you estimate coefficients for whichever basis-functions you're using.

Since they are regularized, are only some regions of the predictor variable associated with coefficients?

No, any x-value has (an implicit set of) coefficients (four for a cubic). Regularization impacts the values of the coefficients (affecting the amount they change as you move across a knot). If you specify the coefficients of the basis function (for whichever basis, such as a B-spline basis, say), you can then work out the coefficients of the corresponding cubics (though it's often not done explicitly, since you can compute fitted values directly from the coefficients of the basis functions, so in many cases you don't need to actually know what each cubic is).

$\endgroup$
2
  • $\begingroup$ I'm having a bit of difficulty working out how estimating the cubic's coefficients directly compared to estimating the coefficients for whichever basis-functions are being used - how do these two differ? $\endgroup$ Jan 26, 2017 at 15:55
  • $\begingroup$ any help here would be greatly appreciated. $\endgroup$ Jan 29, 2017 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.