I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of reasons I shall use SVR for such task.
I have a dataset with dependent variables Y and features X. Suppose for simplicity that both Y and X take real values from -1 to 1. When I plot Y vs X I get a non-trivial and definitely non-linear trend. I fit the dataset with the SVR scikit-learn regressor using a an 'rbf' kernel function
SVR(kernel = 'rbf').fit(X,Y). The result is not bad, but I was wondering how to improve it.
In particular, I notice that within the whole data set Y vs X there are some sub-trend. For example for the range X0<X<X1 the trend is linear, for X1<X<X2 is quadratic etc. where X0,X1,X2 are some values of X.
I was wondering if it is possible to define a different kernel for different range of X values, and then use a linear combination of those kernels to model the whole dataset. In other words something like this:
kernel1 = SVR(kernel = 'rbf').fit(X0_X1, Y0_Y1) kernel2 = SVR(kernel = 'rbf').fit(X1_X2, Y1_Y2) kernel_tot = SVR(kernel = a*kernel1 + b*kernel2 ).fit(X,Y)
where X0_X1 (X1_X2) represents the range of X values between X0 and X1 (X1 and X2), and Y0_Y1 (Y1_Y2) the correspondent Y values.
I expect that kernel1 will well fit the first subset of data (X0_X1), the kernel2 the second (X1_X2) and hopefully the new defined kernel,
a*kernel1 + b*kernel2, will fit the whole dataset with more accuracy than previously (i.e. by using one single rbf kernel for the whole dataset).
I hope the question is clear. I tried to upload images of such dataset to help the description but looks like it is not allowing me to.