I'm trying to use Support Vector Regression (SVR) to model a trend.
I have a dataset with dependent variables $Y$ and features $X$. Suppose for simplicity that both $Y$ and $X$ take real values from $-1$ to $1$. When I plot $Y$ vs $X$, I get a non-trivial and definitely non-linear trend. I fit the dataset with the SVR scikit-learn regressor using an 'rbf' kernel function SVR(kernel = 'rbf').fit(X,Y). The result is not bad, but I want to improve it.
In particular, I notice that within the whole data set $Y$ vs $X$, there are some sub-trend. For example, for the range $X_0<X<X_1$ the trend is linear, for $X_1<X<X_2$ is quadratic etc. where $X_0,X_1,X_2$ are some values of $X$.
Now, I defined a different kernel for different range of $X$ values, and then used a linear combination of those kernels to model the whole dataset. In other words, something like this:
kernel1 = SVR(kernel = 'rbf').fit(X0_X1, Y0_Y1)
kernel2 = SVR(kernel = 'rbf').fit(X1_X2, Y1_Y2)
kernel_tot = SVR(kernel = a*kernel1 + b*kernel2 ).fit(X,Y)
where X0_X1 (X1_X2) represents the range of $X$ values between $X_0$ and $X_1$ ($X_1$ and $X_2$), and Y0_Y1 (Y1_Y2) the correspondending $Y$ values.
I expect that kernel1 will fit the first subset of data (X0_X1) well, the kernel2 the second subset (X1_X2), and the new defined kernel, a*kernel1 + b*kernel2, will fit the whole dataset with more accuracy than previously (i.e. by using one single rbf kernel for the whole dataset).
Is this a valid approach?
