1
$\begingroup$

I am a beginner, and I am looking for some advice regarding the use of Support Vector Regression (SVR) to model (or fit if you prefer) a trend. Before you suggest other methods, for a number of reasons I shall use SVR for such task.

I have a dataset with dependent variables Y and features X. Suppose for simplicity that both Y and X take real values from -1 to 1. When I plot Y vs X I get a non-trivial and definitely non-linear trend. I fit the dataset with the SVR scikit-learn regressor using a an 'rbf' kernel function SVR(kernel = 'rbf').fit(X,Y). The result is not bad, but I was wondering how to improve it.

In particular, I notice that within the whole data set Y vs X there are some sub-trend. For example for the range X0<X<X1 the trend is linear, for X1<X<X2 is quadratic etc. where X0,X1,X2 are some values of X.

I was wondering if it is possible to define a different kernel for different range of X values, and then use a linear combination of those kernels to model the whole dataset. In other words something like this:

kernel1    = SVR(kernel = 'rbf').fit(X0_X1, Y0_Y1) 
kernel2    = SVR(kernel = 'rbf').fit(X1_X2, Y1_Y2) 
kernel_tot = SVR(kernel = a*kernel1 + b*kernel2 ).fit(X,Y) 

where X0_X1 (X1_X2) represents the range of X values between X0 and X1 (X1 and X2), and Y0_Y1 (Y1_Y2) the correspondent Y values.

I expect that kernel1 will well fit the first subset of data (X0_X1), the kernel2 the second (X1_X2) and hopefully the new defined kernel, a*kernel1 + b*kernel2, will fit the whole dataset with more accuracy than previously (i.e. by using one single rbf kernel for the whole dataset).

I hope the question is clear. I tried to upload images of such dataset to help the description but looks like it is not allowing me to.

Best, M.

$\endgroup$
0
$\begingroup$

I think you are over complicating this. You can just filter your feature space and fit two separate models. You can make new kernels but I think you described using different kernels for different parts of your feature space, and I cant find anything supporting that decision.

$\endgroup$
1
  • $\begingroup$ Hi, thanks for your answer. I see what you mean and I guess it makes sense. I could just use two models, one for each range. However, as I am learning now these things I was wondering if what I asked is possible, as I think is a more general solution. If nothing else, I am curious about it. $\endgroup$ – Mash Oct 5 '20 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.