I am trying to fit polynomial ridge regression line for some datapoints. However I know that most of new data will be over the bounds of training data. Because of this fact I want to make part of the data in the right end more important to model than data on the left side so for the model it is more important to fit closely the data on the right than on the left even on expense of general error. How do I achieve such effect apart from resampling data from the right side? I would like to achieve smoothe rising importance which is hard to make just by resampling.
2
-
$\begingroup$ Extrapolation of polynomials outside the data is often risky $\endgroup$– HenryJun 13, 2020 at 16:47
-
$\begingroup$ Which method should I use then? $\endgroup$– SlajniJun 13, 2020 at 17:25
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In principle, there's no problem with differential weighting of data points in a regression to serve some useful purpose. Weighting data values inverse to estimated variance is an example. Regression software typically allows for point-by-point weighting. You're not restricted to any particular formula for weights, so you could impose weights based on the distances of the data points from the boundary of interest in your the training data.
But as the Great Yogi allegedly said*:
It's tough to make predictions, especially about the future.
So there's a danger in weighting to extrapolate beyond the bounds of your training data, whether the bounds are in space, time, or covariate values.
It's even a bigger risk with weighting polynomial regressions toward the edges. This answer shows a particularly egregious example: the wiggles in the polynomial fit get even wilder toward the outside edges, the type of region you want to emphasize in your weighting.
Restricted cubic splines can be a better choice for modeling nonlinear relationships. Piecewise cubic polynomials are fit between a set of pre-defined knot positions along the x-axis, subject to the following restriction on how they are pieced together into a combined function: the function's values, its slopes, and its second derivatives must all be continuous at all knot positions. You could still combine that approach with weighting, should you choose, but beyond the outermost knots the spline function simply extrapolates linearly.
If your data set has some inherent structure like periodicity then restricted cubic splines might not be a good choice. Then again, neither would polynomials. In that case you probably should be using something like time-series analysis that takes internal correlations among data points into account.
*This quote might have originated with Niels Bohr, but many more people know of the renowned American philosopher that is cited here.
