I'm working through an example of using cubic spline regression for logistic regression classification from Elements of Statistical Learning [1] (Phoneme classification - Example 5.2.3 on page 148).
The idea of the model is this: The data consists of a transform of two classes of time series and we want to classify these. We can use a logistic regression, treating each of the 256 values of frequency as separate features or we can try to leverage the structure of the timeseries and fit a cubic spline to it, and then regress on that.
It's fairly straightforward to fit the cubic spline model described in python:
from patsy import dmatrix
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
basis = dmatrix(f"cr(x, df=12) - 1", {"x": range(256)})
smoothed_lr = Pipeline(
[
FunctionTransformer(lambda X: basis @ X, validate=True),
LogisticRegression(),
]
).fit(X_train, y_train)
I was interested to see if I could use a generalisation of those cubic splines to similar effect - Chapter 9 of [1] suggests that a wider class of smoothing functions are available for use in GAMs - but from some experimentation with the pygam package I can only seem to fit one spline for each of the 256 'features' separately. Anyone got any pointers?
